Polarization Image Fusion Research Articles

Research on the Processing of Image and Spectral Information in an Infrared Polarization Snapshot Spectral Imaging System

In order to solve the problems of a low target recognition rate and poor real-time performance brought about by conventional infrared imaging spectral detection technology under complex background conditions or in the detection of targets of weak radiation or long distance, a kind of infrared polarization snapshot spectral imaging system (PSIFTIS) and a spectrum information processing method based on micro-optical devices are proposed in this paper, where the synchronous acquisition of polarization spectrum information is realized through the spatial modulation of phase with a rooftop-shaped multi-stage micro-mirror and the modulation of the polarization state of light with a micro-nanowire array. For the polarization interference image information obtained, the infrared polarization spectrum decoupling is realized by image segmentation, optical path difference matching, and image registration methods, the infrared polarization spectrum reconstruction is realized by Fourier transform spectral demodulation, and the infrared polarization image fusion is realized by decomposing and reconstructing the high- and low-frequency components of the polarization image based on the Haar wavelet transform. The maximum spectral peak wavenumber error of the four polarization channels of the polarization spectrum reconstruction is less than 2 cm−1, and the polarization angle error is within 1°. Ultimately, compared with the unprocessed polarization image unit, the peak signal-to-noise ratio is improved by 45.67%, the average gradient is improved by 8.03%, and the information entropy is improved by 56.98%.

Polarization image fusion method based on polarization saliency with generator adversarial network

Aiming at the problem that the polarization image fusion process does not make good use of the polarization imaging characteristics, we propose a polarization image fusion method based on polarization saliency. On the one hand, we propose a new definition of physical property characterization based on polarization properties:polarization saliency. We construct it from three perspectives: image difference, position, and contrast. On the other hand, we propose a polarization saliency based fusion network (PSGGAN). The information decision block is constructed to enhance the polarization salient information and texture detail information of the source image. The loss function based on polarization saliency is designed to regulate the pixel distribution constraints and the degree of polarization information preservation. Qualitative and quantitative experiments demonstrate the superiority of our PSGGAN over state-of-the-art methods in terms of visual effects and quantitative metrics.

Polarization image fusion method with image enhancement

Because of their complementary characteristics, intensity images and polarization images are often fused to produce information-rich images. However, the polarization characteristics are easily affected by the object’s environment, and the image fusion process may lose important information. In this paper, we propose an unsupervised end-to-end network framework based on a CNN for intensity images and degree of linear polarization images. First, we construct our own polarization dataset to solve the limitations of the training dataset; a hybrid loss function is designed to form an unsupervised learning process; and a Laplace operator enhancement layer is introduced into the network to further improve the quality of the fused images. Subjective and objective comparison experiments prove that the proposed fusion network is visually superior to several classical fusion methods.

Learning based polarization image fusion under an alternative paradigm

Learning based polarization image fusion under an alternative paradigm

DBPFNet: a dual-band polarization image fusion network based on the attention mechanism and atrous spatial pyramid pooling.

In the current image fusion techniques, typically dual-band images are fused to obtain a fused image with salient target information, or intensity and polarization images are fused to achieve an image with enhanced visual perception. However, the current lack of dual-band polarization image datasets and effective fusion methods pose significant challenges for extracting more information in a single image. To address these problems, we construct a dataset containing intensity and polarization images in the visible and near-infrared bands. Furthermore, we propose an end-to-end image fusion network using attention mechanisms and atrous spatial pyramid pooling to extract key information and multi-scale global contextual information. Moreover, we design efficient loss functions to train the network. The experiments verify that the proposed method achieves better performance than the state-of-the-art in both subjective and objective evaluations.

Object detection based on polarization image fusion and grouped convolutional attention network

Object detection based on polarization image fusion and grouped convolutional attention network

Noise-aware infrared polarization image fusion based on salient prior with attention-guided filtering network.

Infrared polarization image fusion integrates intensity and polarization information, producing a fused image that enhances visibility and captures crucial details. However, in complex environments, polarization imaging is susceptible to noise interference. Existing fusion methods typically use the infrared intensity (S0) and degree of linear polarization (DoLP) images for fusion but fail to consider the noise interference, leading to reduced performance. To cope with this problem, we propose a fusion method based on polarization salient prior, which extends DoLP by angle of polarization (AoP) and introduces polarization distance (PD) to obtain salient target features. Moreover, according to the distribution difference between S0 and DoLP features, we construct a fusion network based on attention-guided filtering, utilizing cross-attention to generate filter kernels for fusion. The quantitative and qualitative experimental results validate the effectiveness of our approach. Compared with other fusion methods, our method can effectively suppress noise interference and preserve salient target features.

A dual-weighted polarization image fusion method based on quality assessment and attention mechanisms

This paper proposes a dual-weighted polarization image fusion method based on quality assessment and attention mechanisms to fuse the intensity image (S0) and the degree of linear polarization (DoLP). S0 has high contrast and clear details, and DoLP has an outstanding ability to characterize polarization properties, so the fusion can achieve an effective complementation of superior information. We decompose S0 and DoLP into base layers and detail layers. In the base layers, we build a quality assessment unit combining information entropy, no-reference image quality assessment, and local energy to ensure the fused image has high contrast and clear and natural visual perception; in the detail layer, we first extract depth features using the pre-trained VGG19, then construct an attention enhancement unit combining space and channels, and finally effectively improve the preservation of detail information and edge contours in the fused image. The proposed method is able to perceive and retain polarization image features sufficiently to obtain desirable fusion results. Comparing nine typical fusion methods on two publicly available and own polarization datasets, experimental results show that the proposed method outperforms other comparative algorithms in both qualitative comparison and quantitative analysis.

Underwater polarization image fusion based on NSCT

Underwater polarization image fusion based on NSCT

Polarization image fusion based on grouped densely connected network

Polarization image fusion based on grouped densely connected network

Underwater Target Detection Utilizing Polarization Image Fusion Algorithm Based on Unsupervised Learning and Attention Mechanism

Since light propagation in water bodies is subject to absorption and scattering effects, underwater images using only conventional intensity cameras will suffer from low brightness, blurred images, and loss of details. In this paper, a deep fusion network is applied to underwater polarization images; that is, the underwater polarization images are fused with intensity images using the deep learning method. To construct a training dataset, we establish an experimental setup to obtain underwater polarization images and perform appropriate transformations to expand the dataset. Next, an end-to-end learning framework based on unsupervised learning and guided by an attention mechanism is constructed for fusing polarization and light intensity images. The loss function and weight parameters are elaborated. The produced dataset is used to train the network under different loss weight parameters, and the fused images are evaluated based on different image evaluation metrics. The results show that the fused underwater images are more detailed. Compared with light intensity images, the information entropy and standard deviation of the proposed method increase by 24.48% and 139%. The image processing results are better than other fusion-based methods. In addition, the improved U-net network structure is used to extract features for image segmentation. The results show that the target segmentation based on the proposed method is feasible under turbid water. The proposed method does not require manual adjustment of weight parameters, has faster operation speed, and has strong robustness and self-adaptability, which is important for research in vision fields, such as ocean detection and underwater target recognition.

Contrast enhanced color polarization image fusion

Polarization image fusion is commonly employed to improve the quality of images in various fields such as object recognition and detection. There have been limited investigations on color-based polarization image fusion methods, which predominantly rely on pseudo-color approaches and are primarily suited for human perception. We propose a contrast-enhanced method for color polarization image fusion based on local contrast and multiscale decomposition to effectively reflect the multidimensional information and assist subsequent vision tasks. The method created a basic local contrast map from the luminance channel of intensity and grayscale map of the degree of linear polarization (DoLP), then extracted high-frequency features to fuse with the local contrast map utilizing rolling fast guided filtering and least squares. Finally, the local contrast map and high-frequency features of DoLP were combined and injected into the color-intensity image. The quantitative and qualitative evaluation based on nine color image fusion methods and eight blind image assessment measures revealed that the proposed algorithm not only achieved image contrast improvement and effective information fusion, but also preserved the color information.

Underwater polarization image fusion based on unsupervised learning and attention mechanisms

Underwater polarization image fusion based on unsupervised learning and attention mechanisms

Design of intensity image and polarization image fusion network

Design of intensity image and polarization image fusion network

Semantic-guided polarization image fusion method based on a dual-discriminator GAN.

Polarization image fusion is the process of fusing an intensity image and a polarization parameter image solved by Stokes vector into a more detailed image. Conventional polarization image fusion strategies lack the targeting and robustness for fusing different targets in the images because they do not account for the differences in the characterization of the polarization properties of different materials, and the fusion rule is manually designed. Therefore, we propose a novel end-to-end network model called a semantic guided dual discriminator generative adversarial network (SGPF-GAN) to solve the polarization image fusion problem. We have specifically created a polarization image information quality discriminator (PIQD) block to guide the fusion process by employing this block in a weighted way. The network establishes an adversarial game relationship between a generator and two discriminators. The goal of the generator is to generate a fused image by weighted fusion of each semantic object of the image, the dual discriminator's objective is to identify specific modalities (polarization/intensity) of various semantic targets. The results of qualitative and quantitative evaluations demonstrate the superiority of our SGPF-GAN in terms of visual effects and quantitative measures. Additionally, using this fusion approach to transparent, camouflaged hidden target detection and image segmentation can significantly boost the performance.

Multi-angle orthogonal differential polarization characteristics and application in polarization image fusion.

With the wide application of image fusion technology in target detection and other fields, the fusion of polarization images and other intensity images is becoming a research focus. Traditional polarization image fusion includes intensity, degree of linear polarization (DOLP), and angle of polarization (AOP). However, images of DOLP and AOP fusion cannot meet the requirements of outstanding positive characteristics. Therefore, we propose a method to calculate the polarization characteristics image that can reflect the difference of polarization characteristics of different materials. The method and process are as follows: First, the polarization detection angle is divided into several angle intervals, and the orthogonal difference characteristics (ODC) image of each interval is obtained by weighting and accumulating the AOP probability density of the angle in the interval and the correlation between images. Second, the ODC images are reconstructed in the gradient domain, and the multi-angle orthogonal differential polarization characteristics (MODPC) image is obtained. The MODPC image is fused with the visible intensity image, and the fusion results are evaluated by using image evaluation indexes such as contrast (C), average gradient (AG), image entropy (E), and peak signal-to-noise ratio (PSNR). The experimental results show that the MODPC and S 0 fusion result images are superior to the DOLP and S 0 fusion results in terms of subjective visual perception and objective indicators among the six classical fusion algorithms. The proposed MODPC image can be applied in target detection.

A Polarization Image Fusion Approach Using Local Energy and MDLatLRR Algorithm

A Polarization Image Fusion Approach Using Local Energy and MDLatLRR Algorithm

TIPFNet: a transformer-based infrared polarization image fusion network

The fusion of infrared intensity and polarization images can generate a single image with better visible perception and more vital information. Existing fusion methods based on a convolutional neural network (CNN), with local feature extraction, have the limitation of fully exploiting salient target features of polarization. In this Letter, we propose a transformer-based deep network to improve the performance of infrared polarization image fusion. Compared with existing CNN-based methods, our model can encode long-range features of infrared polarization images to obtain global contextual information using the self-attention mechanism. We also design a loss function with the self-supervised constraint to boost the performance of fusion. Experiments on the public infrared polarization dataset validate the effectiveness of the proposed method. Our approach achieves better fusion performance than the state-of-the-art.

Infrared polarization and intensity image fusion method based on multi-decomposition LatLRR

With the development of infrared imaging technology, the study of infrared polarization and infrared intensity image fusion has great potential for application. In this paper, we propose a fusion method based on multi-decomposition latent low-rank representation (LatLRR) and combine it with dual- simplified pulse coupled neural network (D-SPCNN) for the improvement of detail layer fusion. First, the source images are decomposed into the low-rank part and saliency part using LatLRR, and the weight map of the low-rank part is calculated as the final base layer using the luminance weighting strategy. Then, the saliency part is decomposed by LatLRR again to get the detail layers after multiple decompositions, and the saliency part is used as the input of D-SPCNN to output the activated detail weight map. After that, the final detail layer is obtained by summing the results of the multiple times detail weighting strategy. Finally, the detail layer and the base layer are combined and reconstructed to get the fused image. Our method has an excellent effect of improving brightness, while retaining sufficient image details. The experimental results show that our proposed method obtains better performance in both objective evaluation and subjective metrics when compared with other methods.

Attention-Guided Polarization Image Fusion Using Salient Information Distribution

Polarization image fusion aims to integrate the polarization information and intensity images. Existing methods ignore the modal variance and salient information differences between polarization and intensity images. Their fusion results are typically damaged by preserving the redundant information in the polarization image. In this paper, we propose a novel unsupervised polarization and intensity image fusion network via pixel information guidance and attention mechanism, named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">PAPIF</i> . We define the information expected to be fused as highly polarized targets in polarization images and rich textures in intensity images. To enhance the preservation of different types of salient information at the pixel level, we design a loss function to perform the pixel distribution constraint between the fused image and source images. Moreover, aiming at the problem that the distributions of polarization information and textures are not consistent, we introduce the attention mechanism into the fusion module. The channel and spatial attention mechanisms fuse and retain salient information and suppress redundant information. Thus, our fusion result can exhibit richer polarization information with more appropriate brightness than existing methods. Experiments on both RGB and gray datasets demonstrate the superiority of our method over the state-of-the-art methods both qualitatively and quantitatively. Our code is publicly available at https://github.com/hanna-xu/PAPIF.