Multi-source Images Research Articles

Review of Image Fusion and its techniques

This paper shows that image fusion is a energetic area in digital image processing. The main goal of image fusion is always to merge information from multiple images of the exact same view in order to deliver only the useful information. It discuss the different image fusion techniques by utilizing these techniques original images were decomposed into low frequency sub band coefficients and band pass direction sub band coefficients based on the non-sub-sampled contour let transform whereas various maps of visual salient features are created by visual salient features the local energy, the contrast as well as the gradient .Low-frequency sub band coefficients are got by utilizing these visual saliency maps. The literature survey is conducted on various recent techniques of image fusion. The survey shows that although present NSCT turns based fusion outperforms over accessible techniques.

Space target image fusion method based on image clarity criterion

Optical and infrared imaging is often used in ground-based optical space target observation. The fusion of the two types of images for a more detailed observation is the key problem to be solved. A space target multimodal image fusion scheme based on the joint sparsity model, which takes the correlations among the native sparse characteristics of the image, clarity features of the image, and multisource images into consideration, is proposed. First, using an overcomplete dictionary, the source images are represented as a combination of a shared sparse component and exclusive sparse components. Second, a method for image clarity feature extraction is proposed to design the fusion rules of exclusive sparse components to obtain the fused exclusive sparse components. Finally, the fused image is reconstructed with the fused sparse components and overcompleted dictionary. The proposed method was tested on the space target image and nature scene image data sets. Compared with traditional methods such as the multiscale transform-based methods, sparse representation-based methods, and joint sparsity representation-based methods, the final experimental results demonstrated that our method outperforms the existing state-of-the-art methods on the human visual effect and the objective evaluation indexes. In particular, for the evaluation indexes Q A B / F and Q E , the scores increase to nearly 10% more than those for traditional methods, which indicates that the fused image of our method has better edge clarity.

A Combined Random Forests and Active Contour Model Approach for Fully Automatic Segmentation of the Left Atrium in Volumetric MRI.

Segmentation of the left atrium (LA) from cardiac magnetic resonance imaging (MRI) datasets is of great importance for image guided atrial fibrillation ablation, LA fibrosis quantification, and cardiac biophysical modelling. However, automated LA segmentation from cardiac MRI is challenging due to limited image resolution, considerable variability in anatomical structures across subjects, and dynamic motion of the heart. In this work, we propose a combined random forests (RFs) and active contour model (ACM) approach for fully automatic segmentation of the LA from cardiac volumetric MRI. Specifically, we employ the RFs within an autocontext scheme to effectively integrate contextual and appearance information from multisource images together for LA shape inferring. The inferred shape is then incorporated into a volume-scalable ACM for further improving the segmentation accuracy. We validated the proposed method on the cardiac volumetric MRI datasets from the STACOM 2013 and HVSMR 2016 databases and showed that it outperforms other latest automated LA segmentation methods. Validation metrics, average Dice coefficient (DC) and average surface-to-surface distance (S2S), were computed as 0.9227 ± 0.0598 and 1.14 ± 1.205 mm, versus those of 0.6222–0.878 and 1.34–8.72 mm, obtained by other methods, respectively.

Coupled Dictionary Learning for Change Detection From Multisource Data

With the increase of multisource data available from remote sensing platforms, it is demanding to develop unsupervised techniques for change detection from multisource data. The difference in imaging mechanism makes it difficult to carry out a direct comparison between multisource data in original observation spaces. Different sensors provide different descriptions on the same truth in low-dimension observation spaces, but the same truth indicates the comparability of multisource data in some high-dimensional feature spaces. Inspired by this, we try to solve this problem by transforming multisource data into a common high-dimension feature space. In this paper, an iterative coupled dictionary learning (CDL) model is proposed for multisource image change detection. This model aims to establish a pair of coupled dictionaries, one of which is responsible for the data from one sensor, whereas the other is responsible for the data from another sensor. The atoms from these two coupled dictionaries have a one-to-one correspondence at the same location. Such a property guarantees the transferability of the reconstruction coefficients between bitemporal patch pairs and provides us a desired mechanism to bridge multisource data and highlight changes. The contributions can be summarized as follows: CDL is designed to explore the intrinsic difference of multisource data for change detection in a high-dimension feature space, and an iterative scheme for unsupervised sample selection is proposed to keep the purity of training samples and gradually optimize the current coupled dictionaries. The experimental results have demonstrated the feasibility, effectiveness, and robustness of the proposed framework.

A rapid multi-source shortest path algorithm for interactive image segmentation

This paper addresses the problem of performing multiple objects, interactive image segmentation. Given a small number of pixels (seeds) with predefined labels, we can quickly and accurately determine the closest seed from each unlabeled pixel. By assigning each pixel to the label same as its closest seed, a rapid image segmentation result can be obtained. Since the shortest distance is considered in this paper which directs our attention of segmentation into path planning problem, Dijkstra occurs to mind. Modifying the classical single-source algorithm, a simple yet rapid multi-source Dijkstra (MSD) algorithm is put forward. From both theoretical and experimental aspects, the proposed algorithm performs quite well in resisting noise, and preserving the objects details. Moreover, under the situation of multiple sources, instead of performing Dijkstra several times to obtain the distance from each pixel to each seed and choose the closest seed, the proposed multi-source image segmentation algorithm could determine the closest seed by running Dijkstra only once. Its efficiency, which will not be affected by the number of initial seed settings, maintains the same as the Dijkstra.

Scale-Aware Pansharpening Algorithm for Agricultural Fragmented Landscapes

Remote sensing (RS) has played an important role in extensive agricultural monitoring and management for several decades. However, the current spatial resolution of satellite imagery does not have enough definition to generalize its use in highly-fragmented agricultural landscapes, which represents a significant percentage of the world’s total cultivated surface. To characterize and analyze this type of landscape, multispectral (MS) images with high and very high spatial resolutions are required. Multi-source image fusion algorithms are normally used to improve the spatial resolution of images with a medium spatial resolution. In particular, pansharpening (PS) methods allow one to produce high-resolution MS images through a coherent integration of spatial details from a panchromatic (PAN) image with spectral information from an MS. The spectral and spatial quality of source images must be preserved to be useful in RS tasks. Different PS strategies provide different trade-offs between the spectral and the spatial quality of the fused images. Considering that agricultural landscape images contain many levels of significant structures and edges, the PS algorithms based on filtering processes must be scale-aware and able to remove different levels of detail in any input images. In this work, a new PS methodology based on a rolling guidance filter (RGF) is proposed. The main contribution of this new methodology is to produce artifact-free pansharpened images, improving the MS edges with a scale-aware approach. Three images have been used, and more than 150 experiments were carried out. An objective comparison with widely-used methodologies shows the capability of the proposed method as a powerful tool to obtain pansharpened images preserving the spatial and spectral information.

Game theoretic hypergraph matching for multi-source image correspondences

In this paper, we develop a game theoretic hypergraph matching framework which enables refining feature correspondences of multi-source images. For images obtained from different sources, we commence by characterizing game strategies in terms of coarse correspondences based on feature similarities. In this scenario, mismatches tend to occur because one feature descriptor may exhibit certain ambiguity in characterizing feature similarities for multi-source images. To address this shortcoming, we develop payoffs for independent game players in terms of higher order strategy similarities, and identify strategies which extinguish in an evolutionary game for achieving maximum average payoff. To render a convenient computational framework, we establish an association uniform hypergraph, referred to as the game hypergraph, with each vertex representing a strategy and each uniform hyperedge characterizing a payoff for independent players. We then exploit the Baum–Eagon scheme for computing the maximum average game payoff for Nash equilibrium within the association hypergraph. Our method is invariant to scene scale variation because of the higher order structures characterized by the strategy similarities. Furthermore, our method is computationally more efficient than existing hypergraph matching methods because of coarse correspondences. Experimental results show the effectiveness of our method for refining multi-source feature correspondences.

Efficient image fusion with approximate sparse representation

In this paper, an efficient approximate sparse representation (SR) algorithm with multi-selection strategy is used to solve the image fusion problem. We have shown that the approximate SR is effective for image fusion even if the sparse coefficients are not the sparsest ones possible. A multi-selection strategy is used to accelerate the process of generating the approximate sparse coefficients which are used to guide the fusion of image patches. The relative parameters are also investigated experimentally to further reduce the computational time. The proposed method is compared with some state-of-the-art image fusion approaches on several pairs of multi-source images. The experimental results exhibit that the proposed method is able to yield superior fusion results with less consumption time.

CHANGE DETECTION WITH MULTI-SOURCE DEFECTIVE REMOTE SENSING IMAGES BASED ON EVIDENTIAL FUSION

Remote sensing images with clouds, shadows or stripes are usually considered as defective data which limit their application for change detection. This paper proposes a method to fuse a series of defective images as evidences for change detection. In the proposed method, post-classification comparison process is firstly performed on multi-source defective images. Then, the classification results of all the images, together with their corresponding confusion matrixes are used to calculate the Basic Belief Assignment (BBA) of each pixel. Further, based on the principle of Dempster-Shafer evidence theory, a BBA redistribution process is introduced to deal with the defective parts of multi-source data. At last, evidential fusion and decision making rules are applied on the pixel level, and the final map of change detection can be derived. The proposed method can finish change detection with data fusion and image completion in one integrated process, which makes use of the complementary and redundant information from the input images. The method is applied to a case study of landslide barrier lake formed in Aug. 3rd, 2014, with a series of multispectral images from different sensors of GF-1 satellite. Result shows that the proposed method can not only complete the defective parts of the input images, but also provide better change detection accuracy than post-classification comparison method with single pair of pre- and post-change images. Subsequent analysis indicates that high conflict degree between evidences is the main source of errors in the result. Finally, some possible reasons that result in evidence conflict on the pixel level are analysed.

In vivo MRI based prostate cancer localization with random forests and auto-context model

Prostate cancer is one of the major causes of cancer death for men. Magnetic resonance (MR) imaging is being increasingly used as an important modality to localize prostate cancer. Therefore, localizing prostate cancer in MRI with automated detection methods has become an active area of research. Many methods have been proposed for this task. However, most of previous methods focused on identifying cancer only in the peripheral zone (PZ), or classifying suspicious cancer ROIs into benign tissue and cancer tissue. Few works have been done on developing a fully automatic method for cancer localization in the entire prostate region, including central gland (CG) and transition zone (TZ). In this paper, we propose a novel learning-based multi-source integration framework to directly localize prostate cancer regions from in vivo MRI. We employ random forests to effectively integrate features from multi-source images together for cancer localization. Here, multi-source images include initially the multi-parametric MRIs (i.e., T2, DWI, and dADC) and later also the iteratively-estimated and refined tissue probability map of prostate cancer. Experimental results on 26 real patient data show that our method can accurately localize cancerous sections. The higher section-based evaluation (SBE), combined with the ROC analysis result of individual patients, shows that the proposed method is promising for in vivo MRI based prostate cancer localization, which can be used for guiding prostate biopsy, targeting the tumor in focal therapy planning, triage and follow-up of patients with active surveillance, as well as the decision making in treatment selection. The common ROC analysis with the AUC value of 0.832 and also the ROI-based ROC analysis with the AUC value of 0.883 both illustrate the effectiveness of our proposed method.

分光轴式多源图像融合系统近距离配准方法研究

分光轴式多源图像融合系统近距离配准方法研究

分光轴式多源图像融合系统近距离配准方法研究

分光轴式多源图像融合系统近距离配准方法研究

Multi-Source Image Fusion Based on Wavelet Decomposition and Directional Filter Bank

Multi-Source Image Fusion Based on Wavelet Decomposition and Directional Filter Bank

Multi-source image fusion algorithm based on cellular neural networks with genetic algorithm

Multi-source image fusion is a technology that can make a single image to include more information by fusing two or more source images which are obtained by different sensors from the same scene. How to keep the detail and edge information of the original images is the key to multi-source image fusion. To address this problem, we propose a new approach which base on cellular neural networks and genetic algorithms. In this approach, the source images are inputted to the work which has determined the parameters using genetic algorithms and then we can get the fused image. Compared with other fusion algorithms, the proposed approach could fuse multi-source images adaptively and maintain the edges and details information effectively.

Superpixel based fusion and demosaicing for multi-focus Bayer images

In this paper, a novel superpixel based multi-focus image fusion algorithm for raw images of single-sensor color imaging devices which incorporates the Bayer pattern is proposed. The proposed algorithm is more efficient than traditional fusion schemes since the raw Bayer pattern images are fused before color demosaicing. With the proposed fusion algorithm, the interpolation errors introduced by the repeated demosaicing operation on multi-source images can also be greatly reduced. In detail, a clarity measurement of Bayer pattern image is defined to judge the focus-level of raw Bayer pattern images, and the fusion operator is performed on superpixels which provide powerful grouping cues of local image feature. By comparing the clarities of superpixels, a weight map is constructed and the guided filter is utilized to refine the weight map. The raw images are merged with refined weight map to get the fused Bayer pattern image, which is restored by the demosaicing algorithm to get the full resolution color image. Experimental results demonstrate that the proposed algorithm can obtain better fused results with more natural appearance and fewer artifacts than the traditional algorithms.

An automatic optical and SAR image registration method with iterative level set segmentation and SIFT

Although optical image registration methods have been successfully developed over the past decades, the registration of optical and synthetic aperture radar (SAR) images is still a challenging problem in remote sensing. Feature-based methods are considered to be more effective for multi-source image registration. However, almost all of these methods rely on the feature extraction algorithms. In this article, a simultaneous segmentation and feature-based registration method based on an iterative level set and scale-invariant feature transform (ILS-SIFT) is proposed. The core idea consists of three aspects: (1) an iterative procedure that combines image segmentation and matching is proposed to avoid registration failure caused by poor feature extraction; (2) a uniform level set segmentation model for optical and SAR images is presented to segment conjugate features; and (3) an improved SIFT algorithm is employed to determine whether the registration was successful. Experimental results have shown the effectiveness and universality of the proposed method.

A fast and high accuracy registration method for multi-source images

Aiming at the problem of low registration accuracy and poor real-time performance which are caused by traditional SIFT and SURF algorithm in multi-source image registration, a fast and high accuracy registration method is proposed. It makes full use of the flexibility of NSCT for image decomposition and the accuracy of SURF for feature location, as well as the quickness of SURF for feature extraction. Firstly, two images are decomposed by NSCT and SURF features are extracted in low frequency images. Secondly, feature points are matched bilaterally by using the ratio of the closest neighbor and second closest neighbor to avoid one-to-many matching. Then RANSAC is applied to refine the matching. Finally, image registration is realized quickly and accurately. Experimental results show that the proposed method is better than traditional SIFT and SURF in registration accuracy and computation time. It has the validity.

Image Fusion based on Local Pixel Information with Rough-Fuzzy C-means Approach

With the increased development of technology, it is necessary to retrieve information from multi source images in order to produce a high quality fused image with spatial and spectral information. Image Fusion is a process which allows the combination of the relevant information from a set of images into a single image where the resultant fused image will be more informative than any of the input images. Though the fused image can have complementary spatial and spectral resolution characteristics, the existing image fusion techniques can distort the spectral information of the multispectral data while merging. In this Paper, a rough set theory based fuzzy c-means approach is introduced for image fusion. The distribution of the local information and spatial constraint affect the damping extent of the pixels in neighbors. With the weighted rough and fuzzy factors depends on the space distance of all the neighboring pixels and their graylevel difference accurately measure the variance and enhance its robustness to noise and outliers.

Study on Multi-objective Optimization Problem of Multi-source Image Fusion

Multi-source image fusion will integrate multiple images derived from the same scene or target collected into a new image to obtain more accurate and more complete description about the scene or target. The multi-objective optimi- zation problem of multi-source image fusion is researched in transform domain. Based on the analysis of multi-objective optimization theory and algorithms, an adaptive differential evolution algorithm is proposed. With adaptive variance fac- tor, dynamical crossover probability function and optimal elite ordering strategy, the algorithm reflects not only good search capability but also good convergence. When applied to multi-objective optimization of multi-source image fusion of transform domain, it will be an effective solution to the comprehensive evaluation in the image fusion process.

Compressed Sensing of a Remote Sensing Image Based on the Priors of the Reference Image

Basic compressed-sensing algorithms for image reconstructions mainly deal with the computation of sparse regularization. Remote sensing applications often have multisource or multitemporal images whose different components are acquired separately. Therefore, this letter considers the reconstruction of a remote sensing image using an auxiliary image from another sensor or another time as the reference. For this application, a new compressed-sensing object function is developed that uses a reference image as a prior. In the new model, the sparsity constraints in the transform domain come from the target image, and the gradient priors in the spatial domain come from the auxiliary reference image. The hybrid regularization is optimized by basing the algorithm on the Bregman split method. The proposed method shows better performances when compared with other three popular compressed-sensing algorithms.