Multi-scale Image Fusion Research Articles

Metallic debossed characters industrial online non‐segmentation identification based on improved multi‐scale image fusion enhancement and deep neural network

This paper investigates accurate industry online scene text recognition techniques for metallic debossed characters (MDCs). As MDCs have low contrast with its background and easy to interfere with corrosion rust, oxidate skin or specular reflection and so forth, a multi-scale image fusion algorithm is first proposed to restore the MDCs’ shape and appearance information to enhance the sample's contrast by using the MDCs' depth characteristic with the help of four-directional illumination. With this method, the three most important reflection-related picture quality evaluation standards, namely contrast, saturation, and exposure, can be enhanced simultaneously. Next, based on the fusion result, a U-shaped network based on VGG-16 network architecture is used for text localization. In order to improve network receptive fields, an improved post-processing output module that combines a text sequence score map for coarsening localization with a single-character score map for fine localization is proposed, so that the network's adaptability and accuracy for long texts are improved. Finally, a convolutional recurrent neural network is adopted to realize the recognition of sequence characters. The advantages and effectiveness of the proposed method are statistically analyzed with the data from a liquefied petroleum gas cylinder annual periodic inspection production line.

Fast Multi-Focus Image Fusion Using Inner Channel Prior

In this paper, we propose a simple but effective sharpness prior—inner channel prior—to detect the focused area of multi-focus images. The inner channel prior is a kind of location feature of natural image sharpness. It is based on a key observation—the high-frequency information of the saturation channel is located inside the object. The inner channel prior indicates the focus degree inside objects, which can obtain better color fidelity and sharper object. Using the proposed prior combined with the multi-scale image fusion, we can directly obtain a sharp image with an extended depth of field focusing on all the objects. Results on a variety of defocused image sequences demonstrate the availability of the proposed prior. In addition, the proposed prior is insensitive to overexposure and can better maintain the color of multi-focus images.

Multi-scale singular value decomposition polarization image fusion defogging algorithm and experiment

The predecessor of China Optics was China Optics and Applied Optics Digest, which was founded in 1985. At that time, it was the only retrieval journal in the field of optics in China. At the end of 2008, China Optics and Applied Optics Digest was renamed

Infrared and visible image fusion using multi‐scale NSCT and rolling‐guidance filter

Image fusion is essential to produce a complementary and comprehensive image, with the source images derived from different sensors, captured from different illumination conditions. In this study, a novel multi-scale image fusion based on the combination of non-subsampled contourlet transform (NSCT) and rolling-guidance filter (RGF) is used to enhance the edges and texture details better than the conventional methods. Initially, infrared (IR) and visible (VIS) source images are multi-scale decomposed to low-frequency and high-frequency sub-band coefficients by NSCT for the best representation of edges and curves. Further, the low-frequency coefficients are decomposed into the base and detail layers by a combination of RGF and GF (Gaussian filter) to retain the features in multiple scales and to reduce halos near the edges. Base layers are fused by saliencybased fusion rule and detail layers are fused by Max absolute rule. High-frequency coefficients are fused by consistency verification based fusion rule to preserve visual details and to suppress noise from source images. Finally, the image is reconstructed by inverse NSCT with good visual perception. Experimental results are evaluated by different evaluation metrics and the results suggest that the proposed method results with better improved source information, clarity and contrast.

Deep learning-based compressed image artifacts reduction based on multi-scale image fusion

One of the visually noticeable compression artifacts in block-based image/video compression platforms is called blocking artifact. Several post-processing methods were presented to reduce such kind of artifacts. However, most methods in the literature often induce visibly blurring artifacts. The paper presents a deep network to eliminate image compression artifacts (usually denoted by image deblocking) based on image fusion in multi-scale manner. Recent deep learning-based related methods usually learn deep models using a loss function in per-pixel manner based on explicit image priors in order to directly produce clean images. In place of existing deep learning-guided approaches, the problem is reformulated in this paper to the learning of the residuals (or artifacts) between the received images and their corresponding clean images (ground truths). In the presented deep framework, an input image is first down-sampled while naturally reducing the blocking artifacts. Then, our multi-scale image fusion model is used for fusing the different down-scaled versions (of less artifacts) with the input image (with severer artifacts) to estimate the blocking artifacts. Then, by deducting the estimated artifacts from the input image, the blocking artifacts can be significantly eliminated and most original image details are preserved simultaneously. The presented method is well applicable to any vision-based computer systems with digital visual codec embedded.

Infrared Image Adaptive Enhancement Guided by Energy of Gradient Transformation and Multiscale Image Fusion

The detail enhancement and dynamic range compression of infrared (IR) images is an important issue and a necessary practical application in the domain of IR image processing. This paper provides a novel approach to displaying high dynamic range infrared images on common display equipment with appropriate contrast and clear detail information. The steps are chiefly as follows. First, in order to protect the weak global details in different regions of the image, we adjust the original normalized image into multiple brightness levels by adaptive Gamma transformation. Second, each brightness image is decomposed into a base layer and several detail layers by the multiscale guided filter. Details in each image are enhanced separately. Third, to obtain the image with global details of the input image, enhanced images in each brightness are fused together. Last, we filter out the outliers and adjust the dynamic range before outputting the image. Compared with other conventional or cutting-edge methods, the experimental results demonstrate that the proposed approach is effective and robust in dynamic range compression and detail information enhancement of IR image.

Underwater Image Restoration Based on Image Fusion

The haziness in underwater images occurs due to two major phenomena namely absorption and scattering of light. Hence, we have proposed an image fusion-based approach to improve the visibility of images obtained underwater. The proposed method uses a single hazy image. Initially the colour corrected and contrast improved versions of the image are obtained. Further, Laplace transform is applied which is followed by replication and saliency mapping on each of the derived images. Multi-scale image fusion technique has been used to combine the inputs. This enables each of the fused images to contribute the most essential feature to obtain the resultant image. Thus, the proposed method significantly restores the quality of the input distorted images

Multiscale modeling algorithm for core images.

Computed tomography (CT) images of large core samples acquired by imaging equipment are insufficiently clear and ineffectively describe the tiny pore structure; conversely, images of small core samples are insufficiently globally representative. To alleviate these challenges, the idea of a super-resolution reconstruction algorithm is combined with that of a three-dimensional core reconstruction algorithm, and a multiscale core CT image fusion reconstruction algorithm is proposed. To obtain sufficient image quality with high resolution, a large-scale core image is used to provide global feature information as well as information regarding the basic morphological structure of a large-scale pore and particle. Then the texture pattern and the tiny pore distribution information of a small-scale core image is used to refine the coarse large-scale core image. A blind image quality assessment is utilized to estimate the degradation model of core images at different scales. A multilevel pattern mapping dictionary containing local binary patterns is designed to speed up the pattern matching procedure, and an adaptive weighted reconstruction algorithm is designed to reduce the blockiness. With our method, images of the same core at different scales were successfully fused. The proposed algorithm is extensively tested on microstructures of different rock samples; all cases of the reconstructed results and those of the actual sample were found to be in good agreement with each other. The final reconstructed image contains both large-scale and small-scale information that can provide a better understanding of the core samples and inform the accurate calculation of parameters.

Multiscale Image Matting Based Multi-Focus Image Fusion Technique

Multi-focus image fusion is a very essential method of obtaining an all focus image from multiple source images. The fused image eliminates the out of focus regions, and the resultant image contains sharp and focused regions. A novel multiscale image fusion system based on contrast enhancement, spatial gradient information and multiscale image matting is proposed to extract the focused region information from multiple source images. In the proposed image fusion approach, the multi-focus source images are firstly refined over an image enhancement algorithm so that the intensity distribution is enhanced for superior visualization. The edge detection method based on a spatial gradient is employed for obtaining the edge information from the contrast stretched images. This improved edge information is further utilized by a multiscale window technique to produce local and global activity maps. Furthermore, a trimap and decision maps are obtained based upon the information provided by these near and far focus activity maps. Finally, the fused image is achieved by using an enhanced decision maps and fusion rule. The proposed multiscale image matting (MSIM) makes full use of the spatial consistency and the correlation among source images and, therefore, obtains superior performance at object boundaries compared to region-based methods. The achievement of the proposed method is compared with some of the latest techniques by performing qualitative and quantitative evaluation.

A Survey on Multi-Scale Medical images Fusion Techniques: Brain Diseases

Brain diseases such as degenerative (alzheimer's disease), neoplastic disease (brain tumor like sarcoma, glioma) are considered an interesting topic areas in the medical image fusion diagnosis. Pixel-level image fusion techniques are designed to combine multiple/multi-scale input images into a fused image, which is expected to be more informative for human or machine perception as compared to any of the input images. Since they are difficult to be summarized ; survey paper are characterized by (1) medical image definition , brain diseases challenges , analysis a various techniques for multi-scale image fusion with its own modalities, fusion rule, fusion strategy and dis-advantage ,Whilst used a database of medical images for medical Harvard School (brain diseases) which contains various groups of co-registered multi-modal images including MRI/CT, MRI/PET and PET/SPECT and MRI (T1/T2) images.

Underwater Image Enhancement based on Histogram Manipulation and Multiscale Fusion

Abstract The underwater images are a good source of information which explores idea about sea creatures, to study about seafloor hydrother-mal vents. Low contrast, color distortion and poor visual appearance are the major issues that an underwater image has to undergo. Such problems were caused by dispersion and refraction of light as they penetrate from rarer to denser media. The scattering of light reduces color contrast. The influence of water in underwater images is not only due to scattering but also due to the presence of underwater organisms. Here we introduce an improved method for underwater image enhancement based on the fusion method that is capable to restore accurately underwater images. The proposed work takes a single image as the input and a sequence of operations such as white balancing, gamma correction, sharpening, manipulating weight maps are performed on the input image. Finally multiscale image fusion of the inputs is done to obtain the resultant output. In the initial stage, color distorted input image is white balanced to remove the color casts maintaining a realistic subsea image. In the second stage, CLAHE is performed on the gamma corrected image. CLAHE plays a significant role in luminance enhancement of underwater images. At the same time, histogram equalization is performed on the sharpened image. The weight maps analyze image characteristics that properly specify the spatial pixel relationship. Finally in the last stage, multiscale pyramidal fusion of the inputs and weight maps are performed. The fusion performed here is the Pyramidal fusion. Result analysis depict the improvement of the underwater images using proposed method.

Multimodal Image Fusion in Clinical Research

With the advancement in medical imaging, far reaching changes are perceptible in clinical analysis. Most of the diagnosed medical evaluation is a resultant of imaging or it is in close synchronization with imaging techniques. This promotes the need to closely read, evaluate and collaborate images. Medical image fusion is a technique to collaborate the clinical images acquired from one or many modalities. Multimodal image analysis during fusion well capitalizes the strength of each medical modality. Incorporating the features from multimodal input images thus holds added potential to abet better-quality diagnosis. Medical image fusion is an intricate task specially when high quality fused image, possessing all relevant information and reasonable operating speed is aimed. Many efforts have been undertaken in this field resulting in diverse research approaches. Image fusion can be performed using medical images obtained from single modality or from multiple modalities. This paper has been designed collaborating the work based on multimodal medical images in multiscale image fusion domain. When the same region, organ or a tissue is captured from various different perceptions, complimentary information is maximized and diagnostic value is reinforced. The fusion framework using Mexican Hat wavelet is proposed using adaptive median filtering detailing each executable fusion block. The Fusion techniques, pre and post processing aspects and evaluation mechanisms are illustrated from literature. The drifts of researchers from single processing to multiple processing hybrid techniques are discussed. The medical modality aspects are detailed. These may provide as a valuable reference to understand the image fusion trade-offs comprehensively with future viabilities.

Image Fusion Method Based on Multi-scale Directional Fast Guided Filter and Convolutional Sparse Representation

Background: A multi-scale multidirectional image fusion method is proposed, which introduces the Nonsubsampled Directional Filter Bank (NSDFB) into the multi-scale edge-preserving decomposition based on the fast guided filter. Methods: The proposed method has the advantages of preserving edges and extracting directional information simultaneously. In order to get better-fused sub-bands coefficients, a Convolutional Sparse Representation (CSR) based approximation sub-bands fusion rule is introduced and a Pulse Coupled Neural Network (PCNN) based detail sub-bands fusion strategy with New Sum of Modified Laplacian (NSML) to be the external input is also presented simultaneously. Results: Experimental results have demonstrated the superiority of the proposed method over conventional methods in terms of visual effects and objective evaluations. Conclusion: In this paper, combining fast guided filter and nonsubsampled directional filter bank, a multi-scale directional edge-preserving filter image fusion method is proposed. The proposed method has the features of edge-preserving and extracting directional information.

Automatic Pavement Crack Detection by Multi-Scale Image Fusion

Pavement crack detection from images is a challenging problem due to intensity inhomogeneity, topology complexity, low contrast, and noisy texture background. Traditional learning-based approaches have difficulties in obtaining representative training samples. We propose a new unsupervised multi-scale fusion crack detection (MFCD) algorithm that does not require training data. First, we develop a windowed minimal intensity path-based method to extract the candidate cracks in the image at each scale. Second, we find the crack correspondences across different scales. Finally, we develop a crack evaluation model based on a multivariate statistical hypothesis test. Our approach successfully combines strengths from both the large-scale detection (robust but poor in localization) and the small-scale detection (detail-preserving but sensitive to clutter). We analyze and experimentally test the computational complexity of our MFCD algorithm. We have implemented the algorithm and have it extensively tested on three public data sets, including two public pavement data sets and an airport runway data set. Compared with six existing methods, experimental results show that our method outperforms all counterparts. Specifically, it increases the precision, recall, and F1-measure over the state-of-the-art by 22%, 12%, and 19%, respectively, on one public data set.

Synthesizing mammogram from digital breast tomosynthesis

The purpose of this study is to develop a new method for generating synthesized mammogram (SM) from digital breast tomosynthesis (DBT) and to assess its potential as an adjunct to DBT. We first applied multiscale bilateral filtering to the reconstructed DBT slices to enhance the high-frequency features and reduce noise. A maximum intensity projection (MIP) image was then obtained from the high-frequency components of the DBT slices. A multiscale image fusion method was designed to combine the MIP image and the central DBT projection view into an SM and further enhance the high-frequency features. We conducted a pilot reader study to visually assess the image quality of SM in comparison to full field digital mammograms (FFDM). For each DBT craniocaudal or mediolateral view, a clinical FFDM of the corresponding view was retrospectively collected. Three MQSA radiologists, blinded to the pathological and other clinical information, independently interpreted the SM and the corresponding FFDM side by side marked with the lesion locations. The differences in the BI-RADS assessments of both MCs and masses between SM and FFDM did not achieve statistical significance for all three readers. The conspicuity of MCs on SM was superior to that on FFDM and the BI-RADS assessments of MCs were comparable while the conspicuity of masses on SM was degraded and interpretation on SM was less accurate than that on FFDM. The SM may be useful for efficient prescreening of MCs in DBT but the DBT should be used for detection and characterization of masses.

Bidimensional Multivariate Empirical Mode Decomposition With Applications in Multi-Scale Image Fusion

Empirical mode decomposition (EMD) is a fully data-driven technique designed for multi-scale decomposition of signals into their natural scale components, called intrinsic mode functions (IMFs). When EMD is directly applied to perform fusion of multivariate data from multiple and heterogeneous sources, the problem of uniqueness, that is, different numbers of decomposition levels for different sources, is likely to occur, due to the empirical nature of EMD. Although the multivariate EMD (MEMD) has been proposed for temporal data, which employs real-valued projections along multiple directions on a unit hypersphere in the $n$ -dimensional space to calculate the envelope and the local mean of multivariate signals, in order to guarantee the uniqueness of the scales, its direct usefulness in 2D multi-scale image fusion is still limited, due to its inability to maintain the spatial information. To address this issue, we propose a novel bidimensional MEMD (BMEMD) which directly projects a bidimensional multivariate signal, which is composed of multiple images, on the unit hypersphere in the $n$ -dimensional space. This is achieved via real-valued surface projections and the mean surface is estimated by interpolating the multivariate scatter data so as to extract common spatio-temporal scales across multiple images. Case studies involving texture analysis and multi-focus image fusion are presented to demonstrate the effectiveness of the proposed method.

Single Fog Image dehazing via fast Multi-scale Image Fusion

In this paper, we present a new image dehazing method via fast multi-scale image fusion. It is designed based on a fusion strategy. Instead of estimating the exact global atmospheric and the transmission separately as most previous methods did, our method directly constructs initial dehazing images with different exposure through the histogram analysis and L0 gradient minimization with adaptive boundary constraint to improve the visual dehazing effect. Experimental results show that this method outperforms state-of-the-art haze removal methods in terms of both efficiency and the dehazing visual effect.

Multi-exposure fusion for welding region based on multi-scale transform and hybrid weight

The multi-exposure fusion is an effective image enhancement technique for high dynamic range (HDR) scene. In this paper, a novel multi-scale hybrid weight fusion framework is proposed to overcome the inherent defects of detail loss during the reconstruction process. Firstly, a novel hybrid weight method is developed by employing the local weight of a single image, the global weight between different exposure images, and the saliency weight from spectral residual model. Secondly, a new multi-scale hybrid weight image fusion algorithm based on Laplacian pyramid is proposed by applying the hybrid weight at each scale. The advantages of the proposed fusion algorithm over individual weight are analyzed from a theoretical point of view and then experimentally verified with multi-exposure image in welding region. Furthermore, the guided filter is utilized to smooth the reconstruction image, Laplacian pyramid image, and saliency weight maps for all the low dynamic range (LDR) images, which can effectively keep the edge information and reduce artifacts of weld seam region. Finally, by comparing our results comprehensively with other methods subjectively and objectively, the proposed fusion framework is verified that it can obtain better performance.

BEMD image fusion based on PCNN and compressed sensing

Bidimensional empirical mode decomposition (BEMD) is a new method for multi-scale image decomposition. In order to forbid useless information to cause an adverse impact on results and make the process have a better self-adaptability, this paper presents a new multi-scale image fusion method, which combines pulse coupled neural network (PCNN) and compressed sensing, and uses them in the BEMD. At first, BEMD processes the original images decomposed into multiple bidimensional intrinsic mode function (BIMFs) and a residual image. Then after doing compression measurement on each layer of BIMFS, we can get compression measurement coefficients. The coefficients at the same layer do the PCNN image fusion, and we can get measurement sampling BIMFs. And then after measurement sampling BIMFs reconstructed, we can get the final BIMFs. The residual images do the fusion based on entropy weight to get the final residual image. At last, after BEMD inverse transform, the final BIMFs and the final residual image get the result image. Experimental studies have shown that compared with other multi-scale decompositions-based image fusion algorithms, the algorithm in this paper has a better performance in terms of objective criteria and visual appearance.

Multi-scale image fusion through rolling guidance filter

Image fusion is essential in enhancing visual quality by blending complementary images, which are derived from different captured conditions or different sensors in the same scene. The role of image fusion in the Internet of Things has become considerably important in the future. For instance, data captured by multiple visual sensors need further computation or fusion, which is based on a network of making a decision or an analysis. A new image fusion method is proposed by using rolling guidance filter and joint bilateral filter in this paper. First, the saliency maps of two source images are extracted by the Kirsch operator. Subsequently, the two source images are decomposed by rolling guidance filter to obtain multi-scale images. Second, joint bilateral filter and optimal correction are utilized to optimize the saliency maps and obtain the final weight maps. Finally, two fusion rules are used to restore the final fused image. The proposed method not only preserves the details of source images, but also suppresses the artifacts effectively. Experimental results prove that our method generates better effects on both visual perception and objective quantization than traditional methods.