Perform Image Fusion Research Articles

ReLAP-Net: Residual Learning and Attention Based Parallel Network for Hyperspectral and Multispectral Image Fusion

Remote sensing applications require high-resolution images to obtain precise information about the Earth???s surface. Multispectral images have high spatial resolution but low spectral resolution. Hyperspectral images have high spectral resolution but low spatial resolution. This study proposes a residual learning and attention-based parallel network based on residual network and channel attention. The network performs image fusion of a high spatial resolution multispectral image and a low spatial resolution hyperspectral image. The network training and fusion experiments are conducted on four public benchmark data sets to show the effectiveness of the proposed model. The fusion performance is compared with classical signal processing???based image fusion techniques. Four image metrics are used for the quantitative evaluation of the fused images. The proposed network improved fusion ability by reducing the root mean square error and relative dimensionless global error in synthesis and increased the peak signal-to-noise ratio when compared to other state-of-the-art models.

Evaluation of the correlation between trigeminal nerve atrophy and trigeminal neuralgia using multimodal image fusion: A single-center retrospective study

Background3D-Slicer is an open-source medical image processing and visualization software. In the surgical treatment of trigeminal neuralgia, it is commonly used to predict the responsible vessels. However, there are few reports on the use of 3D-Slicer software to quantitatively measure the bilateral trigeminal nerve volume in patients with primary trigeminal neuralgia (PTN) based on the three-dimensional images. Therefore, this study aims to explore the role of three-dimensional fused images processed by 3D-Slicer in the evaluation of trigeminal nerve atrophy, providing an objective basis for the diagnosis of PTN. Methods57 PTN patients who underwent microvascular decompression (MVD) or percutaneous balloon compression (PBC) surgery in Hebei general hospital between January 2020 and April 2023 were included. Additionally, 30 patients with facial spasms(HFS) were included as a control group. All patients underwent 3D-TOF-MRA and 3D-FIESTA sequence examinations. Comparisons of bilateral trigeminal nerve volumes within and between groups were conducted by performing image fusion using 3D-slicer. ResultsThe volume of the affected trigeminal nerve in the MVD group (33.96 mm³±12.61 mm³) and PBC group (23.05 mm³±7.71 mm³) was smaller than that of the unaffected trigeminal nerve in the MVD group (39.61 mm³±12.83 mm³) and PBC group (26.14 mm³±6.42 mm³), as well as the average volume of the trigeminal nerve in the control group (40.27 mm³±10.25 mm³) (P<0.05). The differences in bilateral trigeminal ganglion volume (∆V) was significant between the MVD group (∆V=23.59 %±14.32 %) and the control group (∆V=14.64 %±10.00 %) (P<0.05). There was no statistical difference in the trigeminal nerve volume difference between the MVD group (∆V=23.59 %±14.32 %) and the PBC group (∆V=26.52 %±15.00 %) (P>0.05). ConclusionTrigeminal nerve atrophy is correlated with primary trigeminal neuralgia. 3D-slicer software can quantitatively measure trigeminal nerve volume and assist in the diagnosis of primary trigeminal neuralgia based on the difference in bilateral trigeminal nerve volumes. However, trigeminal nerve atrophy is not associated with postoperative pain recurrence in patients.

Fusion of remote sensing images based on multi-resolution analysis

A fast and reliable fusion algorithm is the key to the practical application of image fusion technology, so the research of the fast fusion algorithm with a good fusion effect is particularly important. Based on the multi-resolution analysis, this paper discusses the principles of the PCNN fusion algorithm, wavelet transform and Laplace pyramid algorithm, and then performs image fusion experiments. Experimental results show that the fusion effect of the wavelet transform and PCNN algorithm is significantly higher than that of the Laplace pyramid algorithm.

Image Fusion using Waveatom Transform for Medical Applications

Image fusion is the process of gathering all the important information of multiple images into a single fused image. It is used in many fields like medical, military, remote sensing etc... Image fusion in a medical field is extensively used by physicians for analysis and treatment, as the fused image contains complementary features present in different medical images obtained from imaging devices. The wave atom transform based medical image fusion is proposed as the potential capabilities of wave atoms have been exploded in many applications like image denoising, fingerprint identification, compression etc. Medical image techniques became important tool for medical diagnosis and treatment. Medical images of different modalities such as MRI, CT, PET, SPECT provide information in a limited domain. In medical resources, MRI and CT images contain structural information whereas PET and SPECT images contain functional Information. Thus, single modality does not provide complete and accurate information, both structural as well as functional information is required. So far, many medical image fusion methods have been developed. The easiest way to perform image fusion is to perform weighted average of input images pixel by pixel. But this method produces side effects such as artifacts &amp; contrast reduction. The proposed method is experimented on various medical images and compared with recent state of fusion methods. Results prove that images obtained from proposed method have better clarity and enhanced information and are practically more helpful for better treatment and quick analysis.

Analysis of Image Fusion Effect in Image Quality of Kidney Stone

Kidney stones are a disease due to the buildup of substances that are not needed in the urinary system. Knowledge of the composition and type of stone is required in the treatment of action. Micro CT is one of the modalities that can be used in determining the composition of kidney stones. However, there are limitations when using single-energy micro-CT. Stone attenuation has almost the same value when using single energy, therefore it is necessary to use dual-energy CT to determine the difference in stone attenuation more precisely. The image of the dual-energy CT micro-energy stone needs to be processed before analyzing and determining the rock composition. Image fusion is one of the image processing techniques that can be used. The purpose of this study was to determine the effect of image fusion on the image quality of five kidney stones. The stages in the research carried out are collecting high energy and low energy projection image data, and performing image fusion on the two projected image data. The results obtained are first, the dual-energy CT image fusion affects the image quality which can be seen from the increase in the signal to noise ratio (SNR) value. A high SNR value provides the best image quality information

Wireless Sensor System of UAV Infrared Image and Visible Light Image Registration Fusion

The application of multisource sensors to drones requires high-quality images to ensure it. In simultaneous interpreting two or more multisensor images based on the same scene or target, the image obtained by the UAV sensor is limited by the imaging time and the shooting angle. The images obtained may not be aligned in the spatial position, thus affecting the fusion effect. Therefore, different sensor images must be registered before image fusion. During the shooting process of the drone imaging sensor, imaging angle, and environmental conditions, the obtained various sensor images will have rotation, translation, and other deformations in the spatial position so that they do not reach the spatial position. Therefore, it is impossible to directly perform image fusion directly. Therefore, before the multisensor image fusion, the image registration process must be completed to ensure that the two images are aligned in space. This paper analyzes the principles; based on the principles of the Powell search algorithm and improved walking algorithm, an algorithm combining Powell and improved walking algorithm is proposed. This paper also studies several traditional image neutrosophic fusions. The algorithm combines the fusion optimization algorithm proposed in this paper greatly reduces the calculation speed and improves the performance of the optimization algorithm and success rate.

Fast all-focus image reconstruction method based on light field imaging

To achieve high-quality imaging of all focal planes with large depth of field information, a fast all-focus image reconstruction technology based on light field imaging is proposed: combining light field imaging to collect field of view information, and using light field reconstruction to obtain a multi-focus image source set, using the improved NSML image fusion method performs image fusion to quickly obtain an all-focus image with a large depth of field. Experiments have proved that this method greatly reduces the time consumed in the image fusion process by simplifying the calculation process of NSML, and improves the efficiency of image fusion. This method not only achieves excellent fusion image quality, but also improves the real-time performance of the algorithm.

Development of High-Speed Image Acquisition and Processing System for Real-Time Plasma Control on EAST

In order to realize advanced plasma control based on visible cameras, a high-speed image acquisition and processing system has been developed recently on Experimental Advanced Superconducting Tokamak (EAST). This system is optimized in many ways to achieve high-speed, real-time, low-latency performance, and to load multiple acquisition cards simultaneously. The acquisition rate of this system can be close to 10 000 FPS when the frame size is set as 320 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times240$ </tex-math></inline-formula> and the pixel depth is set as 8 bits. DMA is used for high-speed data transmission; the memory copy function is optimized for reducing the time cost on data memory reading and writing. Besides, the visualization subsystem based on the Python web can communicate with the acquisition machines and also can synthesize data of multiple acquisition machines in real time to perform image fusion and access display. In addition, a thermal event recognition function based on visible imaging is also included in this system. The convolutional neural network (CNN) model of hot spots and multifaceted asymmetric radiation from the edge (MARFE) detection for EAST plasma discharge has been developed so far. The detection results can be visualized in quasi-real time. In terms of data storage, a new data storage format is designed; a GUI for off-line data analysis and processing based on MATLAB is provided.

Performance Evaluation of Change Detection in SAR Images Based on Hybrid Antlion DWT Fuzzy c-Means Clustering

Abstract In this paper, the main objective is to detect changes in the geographical area of Ottawa city in Canada due to floods. Two multi-temporal Synthetic Aperture Radar (SAR) images have been taken to evaluate the un-supervised change detection process. In this process, two ratio operators named as Log-Ratio and Mean-Ratio are used to generate a difference image. Performing image fusion based on DWT by selecting optimum filter coefficients by satisfying the wavelet filter coefficient properties through a novel image fusion technique is named as ADWT. GA, PSO, AntLion Optimization algorithms (ALO) and Hybridized AntLion Algorithm (HALO) have been adapted to perform the ADWT based image fusion. Segmentation has been performed based on fuzzy c-Means clustering to detect changed and unchanged pixels. Finally, the performance of the proposed method will be analysed by comparing the segmented image with the ground truth image in terms of sensitivity, accuracy, specificity, precision, F1-score.

ECG Heartbeat Classification Using Multimodal Fusion

Electrocardiogram (ECG) is an authoritative source to diagnose and counter critical cardiovascular syndromes such as arrhythmia and myocardial infarction (MI). Current machine learning techniques either depend on manually extracted features or large and complex deep learning networks which merely utilize the 1D ECG signal directly. Since intelligent multimodal fusion can perform at the state-of-the-art level with an efficient deep network, therefore, in this paper, we propose two computationally efficient multimodal fusion frameworks for ECG heart beat classification called Multimodal Image Fusion (MIF) and Multimodal Feature Fusion (MFF). At the input of these frameworks, we convert the raw ECG data into three different images using Gramian Angular Field (GAF), Recurrence Plot (RP) and Markov Transition Field (MTF). In MIF, we first perform image fusion by combining three imaging modalities to create a single image modality which serves as input to the Convolutional Neural Network (CNN). In MFF, we extracted features from penultimate layer of CNNs and fused them to get unique and interdependent information necessary for better performance of classifier. These informational features are finally used to train a Support Vector Machine (SVM) classifier for ECG heart-beat classification. We demonstrate the superiority of the proposed fusion models by performing experiments on PhysioNet's MIT-BIH dataset for five distinct conditions of arrhythmias which are consistent with the AAMI EC57 protocols and on PTB diagnostics dataset for Myocardial Infarction (MI) classification. We achieved classification accuracy of 99.7% and 99.2% on arrhythmia and MI classification, respectively. Source code at https://github.com/zaamad/ECG-Heartbeat-Classification-Using-Multimodal-Fusion.

An Image Fusion Algorithm based on Modified Contourlet Transform

Multi-focus image fusion has established itself as a useful tool for reducing the amount of raw data and it aims at overcoming imaging cameras’ finite depth of f ield by combining information from multiple images with the same scene. Most of existing fusion algorithms use the method of multi-scale decompositions (MSD) to fuse the s ource images. MSD-based fusion algorithms provide much better performance than the conventional fusion methods .In the image fusion algorithm based on multi-scale decomposition, how to make full use of the characteristics of coefficients to fuse images is a key problem.This paper proposed a modified contourlet transform(MCT) based on wavelets and nonsubsampled directional filter banks(NSDFB). The image is decomposed in wavelet domain,and each highpass subband of wavelets is further decomposed into multiple directional subbands by using NSDFB. The MCT has the important features of directionality and translation invariance. Furthermore, the MCT and a novel region energy strategy are exploited to perform image fusion algorithm. simulation results shows that the proposed method can the fusion results visually and also improve in objective evaluating parameters.

Implementation of differential evolution algorithm to perform image fusion for identifying brain tumor

Implementation of differential evolution algorithm to perform image fusion for identifying brain tumor

Hyperspectral and Multispectral Image Fusion Techniques

The hyper spectral image covers a broad range of wavelengths in electromagnetic spectrum, spanning from visible to near-infrared region. The basic objective of hyperspectral imaging is to attain the spectrum for each pixel in the image of a scene, with the aim of identifying objects in the scene and its classification. The hyperspectral images give detailed spectral information but their spatial resolution is very poor. So to enhance the visual quality of the hyperspectral image, we can perform image fusion with high spatial information multispectral image. This paper provides a complete description of hyperspectral imaging and image fusion methods of hyperspectral and multispectral images. A quantitative and qualitative comparative analysis on performance of various hyperspectral and multispectral image fusion techniques are also done.

An Enhanced Image Fusion Algorithm by Combined Histogram Equalization and Fast Gray Level Grouping Using Multi-Scale Decomposition and Gray-PCA

Image enhancement is a challenging task in image analysis particularly, it is more challenging in performing image fusion. Image fusion is the process of combining multiple images to produce quality output without any variation in contrast, blurring, and noise. Many image fusion algorithms have been implemented, but their final fused images suffer from variations in background contrast, uneven illumination, blurring, and the presence of noise. To overcome the aforementioned issues, this paper proposed a new image fusion method, which improves image contrast and also gives appropriate details of the image. Our method is based on a set of conventional techniques such as amalgamated histogram equalization and fast gray-scale grouping to handle the problems mentioned, and we improve overall fusion strategies by proposing a novel principal component analysis technique to convert RGB types images to high gray-scale contrast image as the final output image. We have carried out many experiments on different common databases used by various researchers. Our proposed method gives good subjective and objective performances compared to other statuses. Our proposed method can be used in different monitoring applications.

Performance of Spatial Frequency Discrete Wavelet Transform Based Remote Sensing Image Fusion Algorithm

The high sensor cost for producing images with superior spectral and spatial qualities in remote sensing application have led to the development of image fusion algorithms. Image fusion technique combines a Panchromatic image and a Multispectral image with an aim to produce images with excellent spatial and spectral qualities. One of the major factors that affect the performance of any image fusion algorithm is the capability of the algorithm in extracting the spatial and spectral data from the respective images and how effective the so extracted information is blended together. One of the recently developed spectral domain algorithm to perform image fusion in remote sensing applications is Spatial Frequency Discrete Wavelet Transform abbreviated as SFDWT. The excellence of SFDWT image fusion algorithm is already proven better than the prevailing algorithms based on Discrete Wavelet Transform. This paper is coined with an eye to realize the performance of SFDWT based image fusion algorithm with respect to IHS-DWT, which being an enhanced form of a typical DWT based image fusion algorithm. The performance of SFDWT and IHS-DWT based image fusion algorithms will be evaluated by applying both techniques in the fusion of urban images received from Pléiades sensors with 1:4 resolution ratio using qualitative and quantitative image quality assessment methods. The consequence of varying the decomposition level on the quality of the images produced using SFDWT image fusion technique and three variants of IHS-DWT techniques based on substitution, averaging and maximum selection will be also evaluated. From the experimental analysis done using MATLAB simulation, it will be vivid that images obtained using image fusion algorithm based on SFDWT are much better than that obtained using IHS-DWT technique with excellent spatial and spectral qualities.

Image registration method for mobile-device-based multispectral optical diagnostics for buildings.

The recent advances in mobile device hardware and software introduced new opportunities to perform more advanced computation and measurements for buildings. Multispectral optical imaging is a widely recognized technique for building diagnostics, as it is non-destructive, quick, and provides rich spatial-spectral information about the different surfaces for decisions related to building energy performance. However, such implementations require an accurate image fusion. The purpose of this study is to introduce a practical and robust image registration method for mobile-device-based multispectral imaging analysis for building diagnostics. Considering the complicated building geometries including walls, floors, ceilings, roofs, windows, and doors, a new approach based on planar homography is chosen for determining the feature points that are required to perform image fusion between different images. The results obtained are compared against the other available methods, which show that the current method provides multispectral images enhanced with accurate qualitative information, as long as the constraints are satisfied. The results are compared with the state-of-the-art methods. The possible impact of the multispectral imaging to the future of building diagnostics is also discussed.

Deep Learning-based Image Segmentation on Multimodal Medical Imaging.

Multi-modality medical imaging techniques have been increasingly applied in clinical practice and research studies. Corresponding multi-modal image analysis and ensemble learning schemes have seen rapid growth and bring unique value to medical applications. Motivated by the recent success of applying deep learning methods to medical image processing, we first propose an algorithmic architecture for supervised multi-modal image analysis with cross-modality fusion at the feature learning level, classifier level, and decision-making level. We then design and implement an image segmentation system based on deep Convolutional Neural Networks (CNN) to contour the lesions of soft tissue sarcomas using multi-modal images, including those from Magnetic Resonance Imaging (MRI), Computed Tomography (CT) and Positron Emission Tomography (PET). The network trained with multi-modal images shows superior performance compared to networks trained with single-modal images. For the task of tumor segmentation, performing image fusion within the network (i.e. fusing at convolutional or fully connected layers) is generally better than fusing images at the network output (i.e. voting). This study provides empirical guidance for the design and application of multi-modal image analysis.

The Relation between Color Spaces and Compositional Data Analysis Demonstrated with Magnetic Resonance Image Processing Applications

This paper presents a novel application of compositional data analysis methods in the context of color image processing. A vector decomposition method is proposed to reveal compositional components of any vector with positive components followed by compositional data analysis to demonstrate the relation between color space concepts such as hue and saturation to their compositional counterparts. The proposed methods are applied to a magnetic resonance imaging dataset acquired from a living human brain and a digital color photograph to perform image fusion. Potential future applications in magnetic resonance imaging are mentioned and the benefits/disadvantages of the proposed methods are discussed in terms of color image processing.

APPLICATION OF SENSOR FUSION TO IMPROVE UAV IMAGE CLASSIFICATION

Abstract. Image classification is one of the most important tasks of remote sensing projects including the ones that are based on using UAV images. Improving the quality of UAV images directly affects the classification results and can save a huge amount of time and effort in this area. In this study, we show that sensor fusion can improve image quality which results in increasing the accuracy of image classification. Here, we tested two sensor fusion configurations by using a Panchromatic (Pan) camera along with either a colour camera or a four-band multi-spectral (MS) camera. We use the Pan camera to benefit from its higher sensitivity and the colour or MS camera to benefit from its spectral properties. The resulting images are then compared to the ones acquired by a high resolution single Bayer-pattern colour camera (here referred to as HRC). We assessed the quality of the output images by performing image classification tests. The outputs prove that the proposed sensor fusion configurations can achieve higher accuracies compared to the images of the single Bayer-pattern colour camera. Therefore, incorporating a Pan camera on-board in the UAV missions and performing image fusion can help achieving higher quality images and accordingly higher accuracy classification results.

ADAPTIVE CHEBYSHEV FUSION OF VEGETATION IMAGERY BASED ON SVM CLASSIFIER

A novel adaptive image fusion method by using Chebyshev polynomial analysis (CPA), for applications in vegetation satellite imagery, is introduced in this paper. Fusion is a technique that enables the merging of two satellite cameras: panchromatic and multi-spectral, to produce higher quality satellite images to address agricurtural and vegetation issues such as soiling, floods and crop harvesting. Recent studies show Chebyshev polynomials to be effective in image fusion mainly in medium to high noise conditions, as per real-life satellite conditions. However, its application was limited to heuristics. In this research, we have proposed a way to adaptively select the optimal CPA parameters according to user specifications. Support vector machines (SVM) is used as a classifying tool to estimate the noise parameters, from which the appropriate CPA degree is utilised to perform image fusion according to a look-up table. Performance evaluation affirms the approach’s ability in reducing the computational complexity to perform fusion. Overall, adaptive CPA fusion is able to optimize an image fusion system’s resources and processing time. It therefore may be suitably incorporated onto real hardware for use on vegetation satellite imagery.