Infrared and visible image fusion algorithm based on NSCT and improved FT saliency detection.

Infrared and visible light image fusion is a hot topic in the field of image processing, aiming to merge the complementary information from source images to produce a fusion image containing richer information. However, the images used for fusion are often taken under extreme lighting conditions at night, which greatly affects the quality of the visible light images-specifically causing low-light degradation phenomena such as uneven brightness, severe noise amplification, and loss of texture details-and leads to poor fusion results; most existing image fusion algorithms do not take into account the lighting factor. For this reason, we propose a multi-scale end-to-end image enhancement fusion method that enhances the illumination of the image while realizing image fusion, which greatly improves the quality of the fused image. The model is based on the existing autoencoder fusion network design, which includes four parts: visible image encoder, infrared image encoder, decoder and fusion module, and the training is accomplished by a novel three-stage training strategy. In the first stage the visible image enhancement consisting of visible image encoder and decoder is trained, and in the second stage the autoencoder consisting of infrared image encoder and decoder is trained. The fusion module is then trained in the third stage. This staged training strategy not only ensures effective learning of each component, but also provides more accurate and robust performance for image fusion in complex environments. Experimental results on public domain datasets show that our end-to-end fusion network achieves superior visual performance results compared to existing methods.

Conventional High Dynamic Range (HDR) image fusion algorithms generally require two or more original images with different exposure times for synthesis, making them unsuitable for real-time processing scenarios such as video streams. Additionally, the synthesized HDR images have the same bit depth as the original images, which may lead to banding artifacts and limits their applicability in professional fields requiring high fidelity. This paper utilizes a Field Programmable Gate Array (FPGA) to support an image sensor operating in Clear HDR mode, which simultaneously outputs High Conversion Gain (HCG) and Low Conversion Gain (LCG) images. These two images share the same exposure duration and are captured at the same moment, making them well-suited for real-time HDR fusion. This approach provides a feasible solution for real-time processing of video streams. An adaptive adjustment algorithm is employed to address the requirement for high fidelity. First, the initial HCG and LCG images are fused under the initial fusion parameters to generate a preliminary HDR image. Subsequently, the gain of the high-gain images in the video stream is adaptively adjusted according to the brightness of the fused HDR image, enabling stable brightness under dynamic illumination conditions. Finally, by evaluating the read noise of the HCG and LCG images, the fusion parameters are adaptively optimized to synthesize an HDR image with higher bit depth. Experimental results demonstrate that the proposed method achieves a processing rate of 46 frames per second for 2688 × 1520 resolution video streams, enabling real-time processing. The bit depth of the image is enhanced from 12 bits to 16 bits, preserving more scene information and effectively addressing banding artifacts in HDR images. This improvement provides greater flexibility for subsequent image processing tasks. Consequently, the adaptive algorithm is particularly suitable for dynamically changing scenarios such as real-time surveillance and professional applications including industrial inspection.

DCDFusion: A dual-consistency decoupling infrared and visible image fusion algorithm for low-light scenes

TIF: A time-series-based image fusion algorithm

The acquisition of images of road surfaces not only establishes a theoretical foundation for road maintenance by relevant departments but also is instrumental in ensuring the safe operation of highway transportation systems. To address the limitations of traditional road surface image acquisition systems, such as low collection speed, poor image clarity, insufficient information richness, and prohibitive costs, this study has developed a high-speed binocular-vision-based system. Through theoretical analysis, we developed a complete system that integrates hybrid anti-shake technology. Specifically, a hardware device was designed for stable installation at the rear of high-speed vehicles, and a software algorithm was implemented to develop an electronic anti-shake module that compensates for horizontal, vertical, and rotational motion vectors with sub-pixel-level accuracy. Furthermore, a road surface image fusion algorithm that combines the stationary wavelet transform (SWT) and nonsubsampled contourlet transform (NSCT) was proposed to preserve multi-scale edge and textural details by leveraging their complementary multidirectional characteristics. Experimental results demonstrate that the fusion algorithm based on SWT and NSCT outperforms those using either SWT or NSCT alone across quality evaluation metrics such as QAB/F, SF, MI, and RMSE: at 80 km/h, the SF value reaches 4.5, representing an improvement of 0.088 over the SWT algorithm and 4.412 over the NSCT algorithm, indicating that the fused images are clearer. The increases in QAB/F and MI values confirm that the fused road surface images retain rich edge and detailed information, achieving excellent fusion results. Consequently, the system can economically and efficiently capture stable, clear, and information-rich road surface images in real-time under high-speed conditions with low energy consumption and outstanding fidelity.

Abstract This paper presents an improved Optical coherence tomography (OCT) image denoising and fusion algorithm by integrating statistical measures with data analysis techniques. OCT images are often degraded by speckle noise, which obscures structural details and impedes diagnostic accuracy. The method effectively suppresses speckle noise while preserving essential structural details, enhancing overall image quality. The proposed algorithm employs two complementary prefiltering strategies to generate optimized inputs for subsequent processing. And we leverage JS divergence for optimal weight allocation in low-frequency subband fusion, integrate mean, median, and mode disparities of high-frequency subband, dynamically adjusts weights to emphasize clinically relevant edges and textures. Experimental results confirm the robustness and effectiveness of our approach in balancing noise suppression and detail preservation. This research provides a reliable denoising framework that improves the quality of OCT images, contributing to more accurate medical diagnostics and expanding the potential applications of OCT imaging.

The task of medical image fusion involves synthesizing complementary information from different modal medical images, which is of very significant for clinical diagnosis. The existing medical image fusion algorithms overly rely on convolution operations and cannot establish long-range dependencies on the source images. This can lead to edge blurring and loss of details in the fused images. Because the Transformer can effectively model long-range dependencies through self-attention, a novel and effective dual-branch feature enhancement network called TVNet is proposed to fuse multimodal medical images. This network combines Vision Transformer and Convolutional Neural Network to extract global context information and local information to preserve detailed textures and highlight the structural characteristics in source images. Furthermore, to extract the multiscale information of images, an enhancement module is used to obtain multiscale characterization information, and the two branches information are efficiently aggregated at the same time. In addition, a hybrid loss function is designed to optimize the fusion results at three levels of structure, feature, and gradient. Experiment results prove that the performance of the proposed fusion network outperforms seven state-of-the-art methods in both subjective visual effects and objective metrics. Our code is available at https://github.com/sineagles/TVNet.

Improving the Temporal and Spatial Accuracy of the Normalized Difference Vegetation Index (NDVI) Map using Satellite Image Fusion Algorithms

This paper proposes a polarization image fusion algorithm based on CNN, Transformer, and attention mechanisms to address the issue of inaccurate scene representation in visible light intensity images under certain conditions. The algorithm network consists of an encoder, fusion module, and decoder. Constructing residual dense blocks in the encoder helps preserve more feature information, thereby enhancing the network’s stability. Within the fusion module, channel attention mechanisms are integrated into the intensity feature map extraction network to enhance the responsiveness of important feature channels. Simultaneously, spatial attention mechanisms are embedded into the polarization feature map extraction network to capture critical features across different spatial positions. The use of Sobel operators to extract gradient information from shallow feature maps effectively enhances the network’s capability to extract detailed features. These optimization measures collectively improve the utilization of feature maps, thereby enhancing the overall performance of the network in image processing tasks. Experimental results demonstrate that the fused images produced by this algorithm not only achieve optimal values across multiple objective evaluation metrics but also exhibit superior visual quality that aligns better with human visual perception.

Abstract The advancement of computational imaging has propelled multimode fiber systems beyond conventional endoscopy by synergizing optical physics and deep learning to reconstruct images from disordered speckle patterns. However, existing multimode fiber (MMF) imaging predominantly relies on single‐channel transmission, limiting information capacity, imaging quality, and color fidelity. Here, multiplexing‐enhanced computational imaging is proposed to overcome these constraints through wavelength‐division multiplexing and multi‐channel fusion. By integrating inverse transmission matrix optimization, multi‐input Pix2Pix generative adversarial networks, and image fusion algorithms, the system reconstructs high‐fidelity color images from multiplexed speckle patterns encoded by RGB wavelengths in a single MMF. Experimental results demonstrate a structural similarity improvement from 0.649 to 0.799 and a 26.4% reduction in color deviation, achieving parallel data transmission and enhance imaging quality. A wavelength‐speckle relationship model and analyze multiplexing capacity limits are established, showing its extensibility to time‐, space‐, and polarization‐division multiplexing. This framework not only enables ultra‐thin full‐color micro‐endoscopy with superior information density, but also expands to hyperspectral imaging, optical neural networks, and quantum communication.

Infrared and visible image fusion algorithm for fire scene environment perception

Medical image fusion is an image processing method that uses computer technology to integrate medical images of different modalities to achieve synchronous visualization of multiple types of information; in this way, multiple medical images complement each other, increasing the accuracy and completeness of clinical diagnosis and treatment. Multiple medical image fusion algorithms have been developed, but they all have drawbacks. One of these is that they are not always reusable or portable due to issues such as faulty fusion rule design. Image reconstruction leads to a decrease in image quality, as the reconstruction process may lose some original information, and the complexity of transformation algorithms and medical images can easily affect performance and robustness. By merging spectral residual (SR) saliency with total variation decomposition, this paper presents a medical image fusion method that addresses current issues. Using total variation, we first dissect the source images to identify their structural and textural elements. Additionally, SRs are utilized for the extraction of saliency images. Moreover, separate procedures are used to merge the structural, textural, and saliency pictures. Finally, the three fused images are added together to form the final product. In terms of both clarity of detail and information retention, our experimental results show that this approach is superior to competing methods. In addition, our [Formula: see text] increased by 31.76%, and our [Formula: see text] increased by 48.19% compared with the average value of the reference algorithms.

Microscopic-Diffraction Imaging Flow Cytometry (MDIFC) is a high-throughput, stain-free technology that captures paired microscopic and diffraction images of cellular events, utilizing machine learning for the classification of cell subpopulations. However, MDIFC is still hindered by challenges related to limited accuracy, processing speed, and a lack of automation. To address this, we propose a novel approach that integrates image fusion techniques with a deep learning-based classification algorithm. Using budding yeast recognition as a model system, we categorized events into three groups: single cells, budding cells, and aggregated cells. Paired images were fused with varying weight factors to generate a comprehensive training dataset for a VGG-net-based Convolutional Neural Network (CNN). For comparison, Support Vector Machines (SVM) and Random Forests (RF) based on Grey-Level Co-occurrence Matrix (GLCM) features were employed. The results demonstrate that the VGG-net classifier achieved an impressive classification accuracy of 0.98 when trained on a dataset with a fusion weight of 0.2 for microscopic images and 0.8 for diffraction images. Furthermore, it demonstrated a high throughput of 260.42 cells per second, surpassing the performance of GLCM-based methods. These findings suggest that the combination of image fusion and deep learning algorithms significantly improves both the speed and accuracy of cell classification in MDIFC, offering substantial benefits for high-throughput cell analysis in biological and medical applications.

Abstract Tropical cyclones (TCs) are among the most impactful extreme disasters affecting humanity, and TC forecasting has become a crucial research area. Addressing the current issues of low utilization of infrared imagery information and insufficient extraction of domain knowledge, we employ objective techniques to extract convective features related to cloud organization from infrared imagery. These features, along with satellite imagery and historical intensity values, are selected as model inputs. This paper introduces a deep learning model designed for the short‐term prediction of TC intensity in the Northwest Pacific by fusing satellite imagery and convective features (TCISP‐fusion). We developed a spatiotemporal feature extraction module to capture high‐level features from the spatio‐temporal sequences of satellite imagery and convective features. Additionally, we introduced a spatiotemporal feature fusion module to integrate asymmetrically distributed convective features while minimizing information loss during feature extraction. Furthermore, we applied the Laplacian Pyramid Image Fusion algorithm to effectively combine observations from the infrared (IR) and water vapor (WV) channels. This method captures large‐scale cloud system structures and retains small‐scale detailed features, generating high‐contrast fused imagery and reducing the complexity of input data. The TCISP‐fusion model achieves a root mean square error of 10.87 kt for 24‐hr intensity prediction of western North Pacific TCs. Compared to traditional and mainstream methods, our model achieves comparable accuracy while significantly reducing the required human and material resources. The data used ensure real‐time applicability, making it highly valuable for operational applications.

Intuitionistic fuzzy similarity measures (IFSMs) play a significant role in applications involving complex decision-making, pattern recognition, and image processing. Several researchers have introduced different methods of IFSMs, yet these IFSMs fail to provide rational decisions. Therefore, in this research, we present a novel IFSM by considering the global maximum and the minimum differences in membership, non-membership, and hesitancy degrees between two intuitionistic fuzzy sets (IFSs). We show that the proposed IFSM meets the fundamental properties and provide numerical examples to prove its superiority. We implement it to solve pattern recognition problems and demonstrate its applicability and feasibility by using the parameter ‘degree of confidence’ as a performance index. Additionally, an image fusion method using the proposed IFSM is developed in this work. To construct an image fusion algorithm, initially, we employ a two-layer decomposition method based on Gaussian filtering to the source images of different modalities to decompose them into the base subimages and the detailed subimages. Then, we use the proposed IFSM to extract the features of base subimages and define two fusion rules to fuse the base subimages and detailed subimages. Then, we show the applicability of this method by conducting extensive experiments using three different types of medical image datasets. We evaluated the effectiveness of the proposed image fusion method using six metrics: Mean, Standard Deviation, feature mutual information, Spatial Frequency, Average Gradient, and Xydeas. Experimental results reveal that the proposed IFSM and fusion approach achieve superior performance compared to most existing methods.

In microscopic imaging, the key to obtaining a fully clear image lies in effectively extracting and fusing the sharp regions from different focal planes. However, traditional multi-focus image fusion algorithms have high computational complexity, making it difficult to achieve real-time processing on embedded devices. We propose an efficient high-resolution real-time multi-focus image fusion algorithm based on multi-aggregation. we use a difference of Gaussians image and a Laplacian pyramid for focused region detection. Additionally, the image is down-sampled before the focused region detection, and up-sampling is applied at the output end of the decision map, thereby reducing 75% of the computational data volume. The experimental results show that the proposed algorithm excels in both focused region extraction and computational efficiency evaluation. It achieves comparable image fusion quality to other algorithms while significantly improving processing efficiency. The average time for multi-focus image fusion with a 4K resolution image on embedded devices is 0.586 s. Compared with traditional algorithms, the proposed method achieves a 94.09% efficiency improvement on embedded devices and a 21.17% efficiency gain on desktop computing platforms.

Hyperspectral and multispectral remote sensing image fusion is an optimal approach for generating hyperspectral–spatial-resolution images, effectively overcoming the physical limitations of sensors. In transformer-based image fusion methods constrained by the local window self-attention mechanism, the extraction of global information and coordinated contextual features is often insufficient. Fusion that aims to emphasize spatial–spectral heterogeneous characteristics may significantly enhance the robustness of joint representation for multi-source data. To address these issues, this study proposes a hyperspectral and multispectral remote sensing image fusion method based on a retractable spatial–spectral transformer network (RSST) and introduces the attention retractable mechanism into the field of remote sensing image fusion. Furthermore, a gradient spatial–spectral recovery block is incorporated to effectively mitigate the limitations of token interactions and the loss of spatial–spectral edge information. A series of experiments across multiple scales demonstrate that RSST exhibits significant advantages over existing mainstream image fusion algorithms.

Research on X-ray Image Fusion Algorithm for Food Foreign Object Detection

Pituitary adenoma (PA) is a common brain tumor located in a small cavity at the cranial base. It disrupts hormonal balance and compresses the optic nerves, leading to abnormal body growth, sexual dysfunction, vision loss, and mortality if untreated. Its surgical resection is highly challenging due to its small size, heterogeneous structure, deep location, and indistinct interface with surrounding nerves, arteries, and brain tissues. Mechanical properties of tumor tissues play a crucial role in their microstructure, growth, and progression. However, data on the mechanical properties of PA tissues is scarce. This study aims to provide detailed mechanical properties of various PA tissues and demonstrate the differences in stiffness between tumors and brain tissues. The viscoelastic properties and collagen content of postoperative PA tissues (n = 40) and normal human brain white matter (n = 7) were analyzed using invitro nanoindentation and histological staining, respectively. Tumor consistency was also assessed preoperatively via magnetic resonance images (MRIs) and intraoperatively through surgeon feedback. PA tissues exhibited a considerable variation in viscoelastic properties; however, their average stiffness was significantly higher than normal brain white matter (p < 0.05). Tumors with firm consistency showed higher collagen content (29.8% 21.2%) than the soft (9.1% 8.1%) and medium (12.9% 9.7%) consistency tumors, however the correlation with mechanical properties was not strong (r = 0.40, p = 0.01). Strong correlations between preoperative predictions, intraoperative observations, and postoperative measurements emphasize the clinical relevance of these findings. These results underscore the potential of mechanical biomarkers to enhance surgical strategies, improve outcomes, and support applications in diagnosis, development of elastography and elastic image fusion algorithms, as well as in robot-assisted interventions.