Visible Image Registration Research Articles

Registration of visible and near infrared unmanned aerial vehicle images based on Fourier-Mellin transform

The combination of aerial images acquired in the visible and near infrared spectral ranges is particularly relevant for agricultural and environmental survey. In unmanned aerial vehicle imagery, such a combination can be achieved using a set of several embedded cameras mounted close to each other, followed by an image registration step. However, due to the different nature of source images, usual registration techniques based on feature point matching are limited when dealing with blended vegetation and bare soil patterns. Here, another approach is proposed based on image spatial frequency analysis. This approach, which relies on the Fourier-Mellin transform, has been adapted to homographic registration and distortion issues. It has been successfully tested on various aerial image sets, and has proved to be particularly robust and accurate, providing a registration error below 0.3 pixels in most cases.

Infrared and visual image registration based on mutual information with a combined particle swarm optimization – Powell search algorithm

Infrared and visual image registration has widespread applications in the remote sensing and military fields. The use of mutual information has proved effective and successful in the infrared and visual image registration process. Optimization algorithms, such as particle swarm optimization (PSO) or the Powell search method, are often used to find the most appropriate registration parameters. The PSO algorithm has a high global search capacity and the search speed is fast initially, but the main weakness is its poor search performance in the later search stage. The Powell search method has a powerful local search capacity, but the search performance and time requirements are highly sensitive to the initial values. Therefore, in this study, we propose a novel hybrid algorithm, which combines the PSO algorithm and Powell search method. First, the PSO algorithm is used to obtain a registration parameter that is close to the global minimum. Using this result, the Powell search method aims to find a more precision registration parameter. Our experimental results demonstrate that the algorithm can correct the scale, rotation, and translation in an effective manner without requiring an additional optimization algorithm. Our method may be a good solution for registering the infrared and visible images, and it obtains better performance in terms of time and precision compared with traditional method.

Registration of thermal and visible light images of diseased plants using silhouette extraction in the wavelet domain

The joint analysis of thermal and visible light images of plants can help to increase the accuracy of early disease detection. Registration of thermal and visible light images is an important pre-processing operation to perform this joint analysis correctly. In the case of diseased plants, registration using common methods based on mutual information is particularly challenging since the plant texture in the thermal image significantly differs from the corresponding texture in the visible light image. Registration methods based on silhouette extraction are therefore more appropriate. This paper proposes an algorithm for registration of thermal and visible light images of diseased plants based on silhouette extraction. The algorithm is based on a novel multi-scale method that employs the stationary wavelet transform to extract the silhouette of diseased plants in thermal images, in which common gradient-based methods usually fail due to the high noise content. Experimental results show that silhouettes extracted using this method can be used to register thermal and visible light images with high accuracy.

A fast and mobile system for registration of low-altitude visual and thermal aerial images using multiple small-scale UAVs

The use of multiple small-scale UAVs to support first responders in disaster management has become popular because of their speed and low deployment costs. We exploit such UAVs to perform real-time monitoring of target areas by fusing individual images captured from heterogeneous aerial sensors. Many approaches have already been presented to register images from homogeneous sensors. These methods have demonstrated robustness against scale, rotation and illumination variations and can also cope with limited overlap among individual images.In this paper we focus on thermal and visual image registration and propose different methods to improve the quality of interspectral registration for the purpose of real-time monitoring and mobile mapping. Images captured by low-altitude UAVs represent a very challenging scenario for interspectral registration due to the strong variations in overlap, scale, rotation, point of view and structure of such scenes. Furthermore, these small-scale UAVs have limited processing and communication power. The contributions of this paper include (i) the introduction of a feature descriptor for robustly identifying corresponding regions of images in different spectrums, (ii) the registration of image mosaics, and (iii) the registration of depth maps. We evaluated the first method using a test data set consisting of 84 image pairs. In all instances our approach combined with SIFT or SURF feature-based registration was superior to the standard versions. Although we focus mainly on aerial imagery, our evaluation shows that the presented approach would also be beneficial in other scenarios such as surveillance and human detection. Furthermore, we demonstrated the advantages of the other two methods in case of multiple image pairs.

Thermal–visible registration of human silhouettes: A similarity measure performance evaluation

When dealing with the registration of information from different image sources, the de facto similarity measure used is Mutual Information (MI). Although MI gives good performance in many image registration applications, recent works in thermal–visible registration have shown that other similarity measures can give results that are as accurate, if not more than MI. Furthermore, some of these measures also have the advantage of being calculated independently from each image to register, which allows them to be integrated more easily in energy minimization frameworks. In this article, we investigate the accuracy of similarity measures for thermal–visible image registration of human silhouettes, including MI, Sum of Squared Differences (SSD), Normalized Cross-Correlation (NCC), Histograms of Oriented Gradients (HOG), Local Self-Similarity (LSS), Scale-Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), Census, Fast Retina Keypoint (FREAK), and Binary Robust Independent Elementary Feature (BRIEF). We tested the various similarity measures in dense stereo matching tasks over 25,000 windows to have statistically significant results. To do so, we created a new dataset in which one to five humans are walking in a scene in various depth planes. Results show that even if MI is a very strong performer, particularly for large regions of interest (ROI), LSS gives better accuracies when ROI are small or segmented into small fragments because of its ability to capture shape. The other tested similarity measures did not give consistently accurate results.

A Registration Method Based on Image Edge Normalized Correlation

In order to solve the infrared and visible image registration problem, a new edge matching method is proposed. Firstly, the algorithm gets edges using Canny operator, and then computes the image edge normalized correlation value (IENC) which is treated as the matching degree of registration. In order to evaluate the performance of the proposed algorithm, a comparison experiment between the proposed algorithm and the algorithm based on Hausdorff distance is done. Experimental results show that the proposed algorithm is effective, small amount of computation and good stability.

Infrared and visual image point set registration based on sensor parameters and refined CPD algorithm

In order to realize the FLIR and visual image registration effectively,an automatic registration algorithm based on sensor parameters and the refined CPD algorithm was proposed.Firstly,geometric rectification based on the attitude angle and height parameters was carried out to eliminate the rotation and scale discrepancies between the FLIR and visual images.Then the edges of visual image and rectified infrared image were extracted and a refined CPD algorithm was proposed for point set registration,the similarity transformation was adopted for fine image registration.Finally,the experiments on real FLIR data show that the proposed algorithm can realize the registration of infrared and visual images effectively and the registration precision can be around one pixel.

Infrared and Visible Image Registration Base on SIFT Features

In the System of Target Tracking Recognition, infrared sensors and visible light sensors are two kinds of the most commonly used sensors; fusion effectively for these two images can greatly enhance the accuracy and reliability of identification. Improving the accuracy of registration in infrared light and visible light images by modifying the SIFT algorithm, allowing infrared images and visible images more quickly and accurately register. The method can produce good results for registration by infrared image histogram equa-lization, reasonable to reduce the level of Gaussian blur in the pyramid establishment process of sift algorithm, appropriate adjustments to thresholds and limits the scope of direction of sub-gradient descriptor. The features are invariant to rotation, image scale and change in illumination.

Infrared and Visible Image Registration Base on SIFT Features

In the System of Target Tracking Recognition, infrared sensors and visible light sensors are two kinds of the most commonly used sensors; fusion effectively for these two images can greatly enhance the accuracy and reliability of identification. Improving the accuracy of registration in infrared light and visible light images by modifying the SIFT algorithm, allowing infrared images and visible images more quickly and accurately register. The method can produce good results for registration by infrared image histogram equa-lization, reasonable to reduce the level of Gaussian blur in the pyramid establishment process of sift algorithm, appropriate adjustments to thresholds and limits the scope of direction of sub-gradient descriptor. The features are invariant to rotation, image scale and change in illumination.

舰船小目标图像配准算法

When the ship target is far away from infrared/visible compound seeker,the target information and features are very weak which can be extracted from the infrared and visible images,and conventional image registration algorithms are hard to take effect on the registration of infrared and visible image with small target.To solve the problem,an infrared and visible image registration algorithm is presented based on sensor parameters.Firstly,according to the imaging model of compound seeker,the image registration is decomposed to two independent steps(field of view registration and shift registration).Secondly,the field of view for different sensors are registered using the sensor parameters.Finally,the shift registration is performed by matching point located based on horizontal high pass energy image and sea-sky line.The experimental results prove that the method is feasible and effective,and can be used in the actual registration of infrared and visible image with small ship target.

An iterative integrated framework for thermal–visible image registration, sensor fusion, and people tracking for video surveillance applications

In this work, we propose a new integrated framework that addresses the problems of thermal–visible video registration, sensor fusion, and people tracking for far-range videos. The video registration is based on a RANSAC trajectory-to-trajectory matching, which estimates an affine transformation matrix that maximizes the overlapping of thermal and visible foreground pixels. Sensor fusion uses the aligned images to compute sum-rule silhouettes, and then constructs thermal–visible object models. Finally, multiple object tracking uses blobs constructed in sensor fusion to output the trajectories. Results demonstrate the advantage of our proposed framework in obtaining better results for both image registration and tracking than separate image registration and tracking methods.