Registration Of Hyperspectral Images Research Articles

Real-Time Registration of Unmanned Aerial Vehicle Hyperspectral Remote Sensing Images Using an Acousto-Optic Tunable Filter Spectrometer

Differences in field of view may occur during unmanned aerial remote sensing imaging applications with acousto-optic tunable filter (AOTF) spectral imagers using zoom lenses. These differences may stem from image size deformation caused by the zoom lens, image drift caused by AOTF wavelength switching, and drone platform jitter. However, they can be addressed using hyperspectral image registration. This article proposes a new coarse-to-fine remote sensing image registration framework based on feature and optical flow theory, comparing its performance with that of existing registration algorithms using the same dataset. The proposed method increases the structure similarity index by 5.2 times, reduces the root mean square error by 3.1 times, and increases the mutual information by 1.9 times. To meet the real-time processing requirements of the AOTF spectrometer in remote sensing, a development environment using VS2023+CUDA+OPENCV was established to improve the demons registration algorithm. The registration algorithm for the central processing unit+graphics processing unit (CPU+GPU) achieved an acceleration ratio of ~30 times compared to that of a CPU alone. Finally, the real-time registration effect of spectral data during flight was verified. The proposed method demonstrates that AOTF hyperspectral imagers can be used in real-time remote sensing applications on unmanned aerial vehicles.

Registration of hyperspectral images and mass spectrometry data for the correlation of tissue optical spectra and molecular profiles.

Hyperspectral imaging (HSI) is a label-free imaging modality that is emerging for non-invasive detection of various diseases including cancers. HSI provides high-resolution spatial images where each pixel has a spectral curve with numerous wavelength bands from the visible to infrared ranges. The rich spatial and spectral information can be used to discriminate various types of tissues and pathophysiological conditions. However, it can be difficult to explain spectral data with respect to the underline cellular and molecular mechanism. In this study, we developed an approach that registers hyperspectral images and mass spectrometry (MS) data where MS provides tissue molecular profiles. Human prostate tissues that were obtained after prostatectomy were used in the experiments. The whole prostate was first sliced every six mm. A customized hyperspectral surgical microscope was used to acquire HSI data from the sliced tissue. For MS data analysis, the sliced tissue of the prostate was divided into 51 small regions and then processed separately for each region. The immediately adjacent tissue was sliced and processed histologically for H&E staining. The MS molecular profiles were correlated with the hyperspectral images in this study.

BIOPHOTONICS PERSPECTIVES IN DETERMINING THE AGE OF TRAUMATIC HAEMORRHAGE FORMATIONS

Background. This article is devoted to the prospects of assessing the dating of traumatic haemorrhage formation using biophotonics technologies, in particular hyperspectral imaging. Currently, the most common method of determinatin the age of bruising in forensic medicine is macroscopy, the low objectivity of which does not allow to reliably determine the timing of the occurrence of the injury. In order to improve the accuracy, various instrumental research methods, including physical and biochemical ones, have been introduced into the practice of forensic medical experts. However, those who use the color of the bruise as the main characteristic are advantageous. These include photometry methods. Nevertheless, for all their benefits, these methods are not able to determine the spatial location of chromophores in the bruise. Hyperspectral imaging has a great potential as it eliminates this problem, which increases the prospects for studying this issue. Aim. Improving the accuracy of determining the age of traumatic hemorrhage formation using hyperspectral imaging. Materials and Methods. Registration of hyperspectral images of traumatic hemorrhages was carried out using a hyperspectral camera designed in the Biophotonics Laboratory of the Skolkovo Institute of Science and Technology under the lead of Dmitry Gorin. Results. As a result of digital image processing at each wavelength, a map of tissue oxygenation is obtained, whichcan be used to see the distribution of oxy- and deoxyhemoglobin chromophores in the bruise. Conclusions. The method of such an actively developing area of biophotonics as hyperspectral imaging is capable of surpassing the research results of other spectral methods.

An Intensity-Independent Stereo Registration Method of Push-Broom Hyperspectral Scanner and LiDAR on UAV Platforms

Unmanned aerial vehicles (UAVs) equipped with hyperspectral scanners and LiDARs can flexibly acquire rich spectral and geometric information about the observation scene. To combine the complementary advantages of multi-source data, the stereo registration of hyperspectral images and LiDAR data has become one of the hot topics in remote sensing community. However, existing research works are more focused on exploiting intensity information from multi-source data, which is applicable to data acquired on manned vehicle platforms or satellite platforms. For UAV platforms with poor stability and limited load, the low signal-to-noise ratio of LiDAR data and the complex distortion of push-broom images bring great challenges to stereo registration. Under this circumstance, an intensity-independent stereo registration method is proposed in this paper, which is based on the physical model of the integrated system and the sensor detection principles. Specifically, the proposed method utilizes the position and orientation system (POS) to reduce the impact of UAV platform motion on hyperspectral imaging, and projection errors are eliminated by the ray tracing model with aid of LiDAR data. Finally, a virtual ray decomposition model based on geometric features is constructed to realize the stereo registration of hyperspectral images and LiDAR data. Compared with an advanced solution and professional processing software, the proposed method has shown better registration performance on two data of different scenarios.

An Interband Registration Method for Hyperspectral Images Based on Adaptive Iterative Clustering

In the context of the problem of image blur and nonlinear reflectance difference between bands in the registration of hyperspectral images, the conventional method has a large registration error and is even unable to complete the registration. This paper proposes a robust and efficient registration algorithm based on iterative clustering for interband registration of hyperspectral images. The algorithm starts by extracting feature points using the scale-invariant feature transform (SIFT) to achieve initial putative matching. Subsequently, feature matching is performed using four-dimensional descriptors based on the geometric, radiometric, and feature properties of the data. An efficient iterative clustering method is proposed to perform cluster analysis on the proposed descriptors and extract the correct matching points. In addition, we use an adaptive strategy to analyze the key parameters and extract values automatically during the iterative process. We designed four experiments to prove that our method solves the problem of blurred image registration and multi-modal registration of hyperspectral images. It has high robustness to multiple scenes, multiple satellites, and multiple transformations, and it is better than other similar feature matching algorithms.

Distributed programming of a hyperspectral image registration algorithm for heterogeneous GPU clusters

Hyperspectral image registration is a relevant task for real-time applications such as environmental disaster management or search and rescue scenarios. The HYFMGPU algorithm was proposed as a single-GPU high-performance solution, but the need for a distributed version has arisen due to the continuous evolution of sensors that generate images with finer spatial and spectral resolutions. In a previous work, we simplified the programming of the multi-device parts of an initial MPI+CUDA multi-GPU implementation of HYFMGPU by means of Hitmap, a library to ease the programming of parallel applications based on distributed arrays. The performance of that Hitmap version was assessed in a homogeneous GPU cluster. In this paper, we extend this implementation by means of new functionalities added to the latest version of Hitmap in order to support arbitrary load distributions for multi-node heterogeneous GPU clusters. Three different load balancing layouts are tested, which prove that selecting a proper layout affects the performance of the code and how this performance is correlated with the use of the GPUs available in the cluster.

HSI-MSER: Hyperspectral Image Registration Algorithm Based on MSER and SIFT

Image alignment is an essential task in many applications of hyperspectral remote sensing images. Before any processing, the images must be registered. The Maximally Stable Extremal Regions (MSER) is a feature detection algorithm that extracts regions by thresholding the image at different grey levels. These extremal regions are invariant to image transformations making them ideal for registration. The Scale-Invariant Feature Transform (SIFT) is a well-known keypoint detector and descriptor based on the construction of a Gaussian scale-space. This article presents a hyperspectral remote sensing image registration method based on MSER for feature detection and SIFT for feature description. It efficiently exploits the information contained in the different spectral bands to improve the image alignment. The experimental results over nine hyperspectral images show that the proposed method achieves a higher number of correct registration cases using less computational resources than other hyperspectral registration methods. Results are evaluated in terms of accuracy of the registration and also in terms of execution time.

Calibration and Characterization of Hyperspectral Imaging Systems Used for Natural Scene Imagery

Calibration and characterization of focal plane hyperspectral imaging systems play an important role in natural scene imagery. Illumination plays a major role during imaging, as both the camera and electronically tunable filter may suffer low transmission at the ends of the visible spectrum, resulting in a low signal to noise ratio. It is important that the spectral characteristics of the imaging system as well as its geometric properties be well characterized and its radiometric performance known. The aim of this article is to identify the main sources of errors in a common design of focal-plane hyperspectral imaging system and devise ways of compensating for these errors. Calibration and characterization of a focal-plane hyperspectral imaging system include nominal wavelength accuracy analysis. This was carried out by capturing images of a mercury vapour lamp to study principal emission lines in the visible spectrum. The linearity of the hyperspectral imaging system was investigated by recording an input-output function. This was accomplished by comparing signals captured by the hyperspectral imaging system and luminance data recorded using a luminance meter. System noise characterization was done by repeated acquisitions of dark noise images captured under identical conditions. Main meridian analysis was accomplished by obtaining sample edge patches from the centre and near-boundary of hyperspectral image and then constructing edge and line spread functions. The final test image analysis involved verifying system calibration, image correction and compensation algorithms. Results show that with proper calibration and characterization of imaging systems, high quality images are obtained and can be used for research works which include hyperspectral image registration and hyperspectral image recognition for natural scenes.

A multi-device version of the HYFMGPU algorithm for hyperspectral scenes registration

Hyperspectral image registration is a relevant task for real-time applications like environmental disasters management or search and rescue scenarios. Traditional algorithms were not really devoted to real-time performance, even when ported to GPUs or other parallel devices. Thus, the HYFMGPU algorithm arose as a solution to such a lack. Nevertheless, as sensors are expected to evolve and thus generate images with finer resolutions and wider wavelength ranges, a multi-GPU implementation of this algorithm seems to be necessary in a near future. This work presents a multi-device MPI $$+$$ CUDA implementation of the HYFMGPU algorithm that distributes all its stages among several GPUs. This version has been validated testing it for 5 different real hyperspectral images, with sizes from about 80 MB to nearly 2 GB, achieving speedups for the whole execution of the algorithm from 1.18 $$\times $$ to 1.59 $$\times $$ in 2 GPUs and from 1.26 $$\times $$ to 2.58 $$\times $$ in 4 GPUs. The parallelization efficiencies obtained are stable around 86 $$\%$$ and 78 $$\%$$ for 2 and 4 GPUs, respectively, which proves the scalability of this multi-device version.

A BAND SELECTION METHOD FOR HIGH PRECISION REGISTRATION OF HYPERSPECTRAL IMAGE

Abstract. During the registration of hyperspectral images and high spatial resolution images, too much bands in a hyperspectral image make it difficult to select bands with good registration performance. Terrible bands are possible to reduce matching speed and accuracy. To solve this problem, an algorithm based on Cram’er-Rao lower bound theory is proposed to select good matching bands in this paper. The algorithm applies the Cram’er-Rao lower bound theory to the study of registration accuracy, and selects good matching bands by CRLB parameters. Experiments show that the algorithm in this paper can choose good matching bands and provide better data for the registration of hyperspectral image and high spatial resolution image.

Spectral-Spatial Scale Invariant Feature Transform for Hyperspectral Images.

Spectral-spatial feature extraction is an important task in hyperspectral image processing. In this paper we propose a novel method to extract distinctive invariant features from hyperspectral images for registration of hyperspectral images with different spectral conditions. Spectral condition means images are captured with different incident lights, viewing angles, or using different hyperspectral cameras. In addition, spectral condition includes images of objects with the same shape but different materials. This method, which is named spectral-spatial scale invariant feature transform (SS-SIFT), explores both spectral and spatial dimensions simultaneously to extract spectral and geometric transformation invariant features. Similar to the classic SIFT algorithm, SS-SIFT consists of keypoint detection and descriptor construction steps. Keypoints are extracted from spectral-spatial scale space and are detected from extrema after 3D difference of Gaussian is applied to the data cube. Two descriptors are proposed for each keypoint by exploring the distribution of spectral-spatial gradient magnitude in its local 3D neighborhood. The effectiveness of the SS-SIFT approach is validated on images collected in different light conditions, different geometric projections, and using two hyperspectral cameras with different spectral wavelength ranges and resolutions. The experimental results show that our method generates robust invariant features for spectral-spatial image matching.

METHODOLOGY FOR DETERMINING OPTIMAL EXPOSURE PARAMETERS OF A HYPERSPECTRAL SCANNING SENSOR

Abstract. The purpose of the presented research was to establish a methodology that would allow the registration of hyperspectral images with a defined spatial resolution on a horizontal plane. The results obtained within this research could then be used to establish the optimum sensor and flight parameters for collecting aerial imagery data using an UAV or other aerial system. The methodology is based on an user-selected optimal camera exposure parameters (i.e. time, gain value) and flight parameters (i.e. altitude, velocity). A push-broom hyperspectral imager- the Headwall MicroHyperspec A-series VNIR was used to conduct this research. The measurement station consisted of the following equipment: a hyperspectral camera MicroHyperspec A-series VNIR, a personal computer with HyperSpec III software, a slider system which guaranteed the stable motion of the sensor system, a white reference panel and a Siemens star, which was used to evaluate the spatial resolution. Hyperspectral images were recorded at different distances between the sensor and the target- from 5m to 100m. During the registration process of each acquired image, many exposure parameters were changed, such as: the aperture value, exposure time and speed of the camera’s movement on the slider. Based on all of the registered hyperspectral images, some dependencies between chosen parameters had been developed: - the Ground Sampling Distance – GSD and the distance between the sensor and the target, - the speed of the camera and the distance between the sensor and the target, - the exposure time and the gain value, - the Density Number and the gain value. The developed methodology allowed us to determine the speed and the altitude of an unmanned aerial vehicle on which the sensor would be mounted, ensuring that the registered hyperspectral images have the required spatial resolution.

METHODOLOGY FOR DETERMINING OPTIMAL EXPOSURE PARAMETERS OF A HYPERSPECTRAL SCANNING SENSOR

The purpose of the presented research was to establish a methodology that would allow the registration of hyperspectral images with a defined spatial resolution on a horizontal plane. The results obtained within this research could then be used to establish the optimum sensor and flight parameters for collecting aerial imagery data using an UAV or other aerial system. The methodology is based on an user-selected optimal camera exposure parameters (i.e. time, gain value) and flight parameters (i.e. altitude, velocity). A push-broom hyperspectral imager- the Headwall MicroHyperspec A-series VNIR was used to conduct this research. The measurement station consisted of the following equipment: a hyperspectral camera MicroHyperspec A-series VNIR, a personal computer with HyperSpec III software, a slider system which guaranteed the stable motion of the sensor system, a white reference panel and a Siemens star, which was used to evaluate the spatial resolution. <br><br> Hyperspectral images were recorded at different distances between the sensor and the target- from 5m to 100m. During the registration process of each acquired image, many exposure parameters were changed, such as: the aperture value, exposure time and speed of the camera’s movement on the slider. Based on all of the registered hyperspectral images, some dependencies between chosen parameters had been developed: <br><br> - the Ground Sampling Distance – GSD and the distance between the sensor and the target,<br><br> - the speed of the camera and the distance between the sensor and the target,<br><br> - the exposure time and the gain value,<br><br> - the Density Number and the gain value.<br><br> The developed methodology allowed us to determine the speed and the altitude of an unmanned aerial vehicle on which the sensor would be mounted, ensuring that the registered hyperspectral images have the required spatial resolution.

Derivative-based scale invariant image feature detector with error resilience.

We present a novel scale-invariant image feature detection algorithm (D-SIFER) using a newly proposed scale-space optimal 10th-order Gaussian derivative (GDO-10) filter, which reaches the jointly optimal Heisenberg's uncertainty of its impulse response in scale and space simultaneously (i.e., we minimize the maximum of the two moments). The D-SIFER algorithm using this filter leads to an outstanding quality of image feature detection, with a factor of three quality improvement over state-of-the-art scale-invariant feature transform (SIFT) and speeded up robust features (SURF) methods that use the second-order Gaussian derivative filters. To reach low computational complexity, we also present a technique approximating the GDO-10 filters with a fixed-length implementation, which is independent of the scale. The final approximation error remains far below the noise margin, providing constant time, low cost, but nevertheless high-quality feature detection and registration capabilities. D-SIFER is validated on a real-life hyperspectral image registration application, precisely aligning up to hundreds of successive narrowband color images, despite their strong artifacts (blurring, low-light noise) typically occurring in such delicate optical system setups.

Automatic Subpixel Registration for a Tunable Hyperspectral Imaging System

Hyperspectral imagery of the Earth's surface is increasingly being acquired from aerial platforms. The many bands acquired by typical hyperspectral instruments are collected either in a push-broom, scanning, or staring fashion. Staring methods can be used in ground- and aerial-based applications and have the advantage of readily producing coherent images. Staring remote sensing instruments need some form of coregistration to match band-to-band pixel locations because it takes some time for the instrument to acquire images and save them as the aerial platform moves above the target scene. A well-known method for registration is the phase correlation (PC), which may be used to register images to an accuracy of plusmn1 pixels. This letter reports an enhancement to the PC method that allows for subpixel registration of hyperspectral images. The x-y location at which the maximum correlation function occurs is fitted with a cubic interpolation to find the maximum. This method was implemented to recover subpixel rotation and translation accuracy from an airborne hyperspectral imaging system, dubbed the Portable Hyperspectral Tunable Imaging System. Results showed that the approach improves up to 9.5% of the normalized cross correlation between wavebands in comparison with the PC method alone