Hyperspectral System Research Articles

Colorimetrical uncertainty of a hyperspectral imaging microscopy system for assessing whole-slide imaging devices.

A whole-slide imaging (WSI) device is a color medical imaging system whose application in digital pathology is to digitalize stained tissue samples into electronic images for pathologists to diagnose without using a conventional light microscope. Testing the color performance of a WSI device usually implies a color target with known truth that is compared with the device output to estimate color differences. Using stained tissue samples as color targets is challenging because the cellular features cannot be measured with ordinary spectroradiometers unless a hyperspectral imaging microscopy system (HIMS) is used. The goal of this study is to determine the colorimetrical uncertainty of such a reference HIMS that is designed to assess the color performance of WSI devices. A set of optical filters are used for that purpose. The color truth, in terms of spectral transmittance in the visible band, of the optical filters is measured by a reference spectroradiometer. The spectral transmittance is combined with a standard illuminant to generate colorimetrical measures using the CIEXYZ and CIELAB formulas. The differences between the reference HIMS and the reference spectroradiometer are evaluated using the CIE 1976 color difference formulas.

Lightweight Integrated Solution for a UAV-Borne Hyperspectral Imaging System

The rapid development of unmanned aerial vehicles (UAVs), miniature hyperspectral imagers, and relevant instruments has facilitated the transition of UAV-borne hyperspectral imaging systems from concept to reality. Given the merits and demerits of existing similar UAV hyperspectral systems, we presented a lightweight, integrated solution for hyperspectral imaging systems including a data acquisition and processing unit. A pushbroom hyperspectral imager was selected owing to its superior radiometric performance. The imager was combined with a stabilizing gimbal and global-positioning system combined with an inertial measurement unit (GPS/IMU) system to form the image acquisition system. The postprocessing software included the radiance transform, surface reflectance computation, geometric referencing, and mosaic functions. The geometric distortion of the image was further significantly decreased by a postgeometric referencing software unit; this used an improved method suitable for UAV pushbroom images and showed more robust performance when compared with current methods. Two typical experiments, one of which included the case in which the stabilizing gimbal failed to function, demonstrated the stable performance of the acquisition system and data processing system. The result shows that the relative georectification accuracy of images between the adjacent flight lines was on the order of 0.7–1.5 m and 2.7–13.1 m for cases with spatial resolutions of 5.5 cm and 32.4 cm, respectively.

Spectral Illumination System Utilizing Spherical Reflection Optics.

Fluorescence imaging microscopy has traditionally been used because of the high specificity that is achievable through fluorescence labeling techniques and optical filtering. When combined with spectral imaging technologies, fluorescence microscopy can allow for quantitative identification of multiple fluorescent labels. We are working to develop a new approach for spectral imaging that samples the fluorescence excitation spectrum and may provide increased signal strength. The enhanced signal strength may be used to provide increased spectral sensitivity and spectral, spatial, and temporal sampling capabilities. A proof of concept excitation scanning system has shown over 10-fold increase in signal to noise ratio compared to emission scanning hyperspectral imaging. Traditional hyperspectral imaging fluorescence microscopy methods often require minutes of acquisition time. We are developing a new configuration that utilizes solid state LEDs to combine multiple illumination wavelengths in a 2-mirror assembly to overcome the temporal limitations of traditional hyperspectral imaging. We have previously reported on the theoretical performance of some of the aspects of this system by using optical ray trace modeling. Here, we present results from prototyping and benchtop testing of the system, including assembly, optical characterization, and data collection. This work required the assembly and characterization of a novel excitation scanning hyperspectral microscopy system, containing 12 LEDs ranging from 365-425 nm, 12 lenses, a spherical mirror, and a flat mirror. This unique approach may reduce the long image acquisition times seen in traditional hyperspectral imaging while maintaining high specificity and sensitivity for multilabel identification and autofluorescence imaging in real time.

An automatic hyperspectral scanning system for the technical investigations of Chinese scroll paintings

Chinese scroll painting is one of the oldest continuous artistic traditions in the world. These paintings are characterized by brushwork, pigment category and painting style. In order to nondestructively investigate the techniques of large-scale Chinese scroll paintings, an automatic hyperspectral scanning system, composed of a hyperspectral camera, a halogen light source, an automatic scanning platform, data processing software and a reference spectral library, was set up. This system was applied to the study of Portrait of Bazalibudala Arhat, an important Chinese scroll painting, along with macroscopic X-ray fluorescence technique. As a result, sketches and outlines of the patterns were detected. Pigments identified included azurite, malachite, white lead, cinnabar, ochre, gold and carbon black and were mapped to the painting. Furthermore, quantitative maps of pigments were obtained, which provided a vital clue in investigating the painters’ craft practice for the use of pigments.

A Kernel-Driven BRDF Approach to Correct Airborne Hyperspectral Imagery over Forested Areas with Rugged Topography

Airborne hyper-spectral imaging has been proven to be an efficient means to provide new insights for the retrieval of biophysical variables. However, quantitative estimates of unbiased information derived from airborne hyperspectral measurements primarily require a correction of the anisotropic scattering properties of the land surface depicted by the bidirectional reflectance distribution function (BRDF). Hitherto, angular BRDF correction methods rarely combined viewing-illumination geometry and topographic information to achieve a comprehensive understanding and quantification of the BRDF effects. This is in particular the case for forested areas, frequently underlaid by rugged topography. This paper describes a method to correct the BRDF effects of airborne hyperspectral imagery over forested areas overlying rugged topography, referred in the reminder of the paper as rugged topography-BRDF (RT-BRDF) correction. The local viewing and illumination geometry are calculated for each pixel based on the characteristics of the airborne scanner and the local topography, and these two variables are used to adapt the Ross-Thick-Maignan and Li-Transit-Reciprocal kernels in the case of rugged topography. The new BRDF model is fitted to the anisotropy of multi-line airborne hyperspectral data. The number of pixels is set at 35,000 in this study, based on a stratified random sampling method to ensure a comprehensive coverage of the viewing and illumination angles and to minimize the fitting error of the BRDF model for all bands. Based on multi-line airborne hyperspectral data acquired with the Chinese Academy of Forestry’s LiDAR, CCD, and Hyperspectral system (CAF-LiCHy) in the Pu’er region (China), the results applying the RT-BRDF correction are compared with results from current empirical (C, and sun-canopy-sensor (SCS) adds C (SCS+C)) and semi-physical (SCS) topographic correction methods. Both quantitative assessment and visual inspection indicate that RT-BRDF, C, and SCS+C correction methods all reduce the topographic effects. However, the RT-BRDF method appears more efficient in reducing the variability in reflectance of overlapping areas in multiple flight-lines, with the advantage of reducing the BRDF effects caused by the combination of wide field of view (FOV) airborne scanner, rugged topography, and varying solar illumination angle over long flight time. Specifically, the average decrease in coefficient of variation (CV) is 3% and 3.5% for coniferous forest and broadleaved forest, respectively. This improvement is particularly marked in the near infrared (NIR) region (i.e., >750 nm). This finding opens new possible applications of airborne hyperspectral surveys over large areas.

Novel Combined Spectral Indices Derived from Hyperspectral and Laser-Induced Fluorescence LiDAR Spectra for Leaf Nitrogen Contents Estimation of Rice

Spectra of reflectance (Sr) and fluorescence (Sf) are significant for crop monitoring and ecological environment research, and can be used to indicate the leaf nitrogen content (LNC) of crops indirectly. The aim of this work is to use the Sr-Sf features obtained with hyperspectral and laser-induced fluorescence LiDAR (HSL, LIFL) systems to construct novel combined spectral indices (NCIH-F) for multi-year rice LNC estimation. The NCIH-F is in a form of FWs* Φ + GSIs* Φ , where Φ is the Sr-Sf features, and FWs and GSIs are the feature weights and global sensitive indices for each characteristic band. In this study, the characteristic bands were chosen in different ways. Firstly, the Sr-Sf characteristics which can be the intensity or derivative variables of spectra in 685 and 740 nm, have been assigned as the Φ value in NCIH-F formula. Simultaneously, the photochemical reflectance index (PRI) formed with 531 and 570 nm was modified based on a variant spectral index, called PRIfraction, with the Sf intensity in 740 nm, and then compared its potential with NCIH-F on LNC estimation. During the above analysis, both NCIH-F and PRIfraction values were utilized to model rice LNC based on the artificial neural networks (ANNs) method. Subsequently, four prior bands were selected, respectively, with high FW and GSI values as the ANNs inputs for rice LNC estimation. Results show that FW- and GSI-based NCIH-F are closely related to rice LNC, and the performance of previous spectral indices used for LNC estimation can be greatly improved by multiplying their FWs and GSIs. Thus, it can be included that the FW- and GSI-based NCIH-F constitutes an efficient and reliable constructed form combining HSL (Sr) and LIFL (Sf) data together for rice LNC estimation.

Design and performance of a hyperspectral imaging system: Preliminary in vivo spectral reflectance measurements of the human iris.

In this paper, we present a novel hyperspectral system designed for imaging the human iris in vivo. The instrument is based on a tunable light source (from 480 nm to 900 nm) and a monochrome camera. The system can acquire hyperspectral cubes in 4 s, with spatial and spectral resolutions of 11 μm and 20 nm, respectively. The optical characterization of the new instrument is described, and we demonstrate its safety with respect to ANSI standards. Preliminary in vivo measurements in human volunteers are also presented.

Drone-Based Fluorescence Lidar Systems for Vegetation and Marine Environment Monitoring

We have developed two different types of drone-based fluorescence lidar systems for vegetation and marine environment monitoring, both based on violet CW diode lasers. An inelastic hyperspectral Scheimpflug lidar system was used for vegetation profiling combined with fluorescence spectral recordings. A light-weight fluorosensor set for fixed-height recordings was employed for monitoring of marine environments, featuring water Raman signals, algal chlorophyll and strong oil spill fluorescence.

小型高光谱图谱仪与激光雷达及其海洋应用

Ocean is an important part of the earth's ecological environment. Exploration and exploitation of marine resources may easily cause serious damage to ocean, such as large-scale oil spill, pollution and red tide caused by oil and gas exploitation. Hyperspectral imaging technology can obtain both image information and spectral information at the same time, and has important applications in marine in-situ detection. In this paper, some recent works about hyperspectral imagers are reviewed, including a small-scale hyperspectral imager combined with fluorescence technology for the classification of oil spills and the estimation of oil film thickness, a multi-mode hyperspectral marine in-situ detection system (in three modes:common reflection or transmission imaging, telescopic imaging and microscopic imaging) for hyperspectral detection of different algae and spores of some fish infectious disease carriers. Hyperspectral technology combined with lidar technology has great potential in monitoring oil spill, red tide and other marine pollutants. An inelastic hyperspectral Scheimpflug lidar system and a ligh-sheet Scheimpflug lidar system are also reviewed. The former is for the type identification of oil spills through the fluorescence spectrum of oil spills, and the latter is for the detection of the 3D shapes of some manikin, shells and corals with the refraction correction at the air-water interface.

The Design of Focal Plane Splitting Unit in a Hyperspectral Lidar System

A hyperspectral lidar system with focal plane splitting unit is proposed, including an off-axis receiving telescope, a grating spectrometer, and a single-tube detector array. The spectrum of the system covers 380-930nm, and is separated by grating spectrometer. The microlens-fiber coupling system guides the echo signal of 50 channels into each detector. The system solves the data processing problem of the bandwidth and gain the line array and area array detector in traditional hyperspectral lidar. And it also meets the requirement of the high efficiency splitting coupling and weak signal acquisition and detection.

Automated Georectification and Mosaicking of UAV-Based Hyperspectral Imagery from Push-Broom Sensors

Hyperspectral systems integrated on unmanned aerial vehicles (UAV) provide unique opportunities to conduct high-resolution multitemporal spectral analysis for diverse applications. However, additional time-consuming rectification efforts in postprocessing are routinely required, since geometric distortions can be introduced due to UAV movements during flight, even if navigation/motion sensors are used to track the position of each scan. Part of the challenge in obtaining high-quality imagery relates to the lack of a fast processing workflow that can retrieve geometrically accurate mosaics while optimizing the ground data collection efforts. To address this problem, we explored a computationally robust automated georectification and mosaicking methodology. It operates effectively in a parallel computing environment and evaluates results against a number of high-spatial-resolution datasets (mm to cm resolution) collected using a push-broom sensor and an associated RGB frame-based camera. The methodology estimates the luminance of the hyperspectral swaths and coregisters these against a luminance RGB-based orthophoto. The procedure includes an improved coregistration strategy by integrating the Speeded-Up Robust Features (SURF) algorithm, with the Maximum Likelihood Estimator Sample Consensus (MLESAC) approach. SURF identifies common features between each swath and the RGB-orthomosaic, while MLESAC fits the best geometric transformation model to the retrieved matches. Individual scanlines are then geometrically transformed and merged into a single spatially continuous mosaic reaching high positional accuracies only with a few number of ground control points (GCPs). The capacity of the workflow to achieve high spatial accuracy was demonstrated by examining statistical metrics such as RMSE, MAE, and the relative positional accuracy at 95% confidence level. Comparison against a user-generated georectification demonstrates that the automated approach speeds up the coregistration process by 85%.

Multichannel correlation improves the noise tolerance of real-time hyperspectral microimage mosaicking.

.Live-subject microscopies, including microendoscopy and other related technologies, offer promise for basic biology research as well as the optical biopsy of disease in the clinic. However, cellular resolution generally comes with the trade-off of a microscopic field-of-view. Microimage mosaicking enables stitching many small scenes together to aid visualization, quantitative interpretation, and mapping of microscale features, for example, to guide surgical intervention. The development of hyperspectral and multispectral systems for biomedical applications provides motivation for adapting mosaicking algorithms to process a number of simultaneous spectral channels. We present an algorithm that mosaics multichannel video by correlating channels of consecutive frames as a basis for efficiently calculating image alignments. We characterize the noise tolerance of the algorithm by using simulated video with known ground-truth alignments to quantify mosaicking accuracy and speed, showing that multiplexed molecular imaging enhances mosaic accuracy by leveraging observations of distinct molecular constituents to inform frame alignment. A simple mathematical model is introduced to characterize the noise suppression provided by a given group of spectral channels, thus predicting the performance of selected subsets of data channels in order to balance mosaic computation accuracy and speed. The characteristic noise tolerance of a given number of channels is shown to improve through selection of an optimal subset of channels that maximizes this model. We also demonstrate that the multichannel algorithm produces higher quality mosaics than the analogous single-channel methods in an empirical test case. To compensate for the increased data rate of hyperspectral video compared to single-channel systems, we employ parallel processing via GPUs to alleviate computational bottlenecks and to achieve real-time mosaicking even for video-rate multichannel systems anticipated in the future. This implementation paves the way for real-time multichannel mosaicking to accompany next-generation hyperspectral and multispectral video microscopy.

Ground-Based Hyperspectral Image Surveillance Systems for Explosive Detection: Part II—Radiance to Reflectance Conversions

The usage of white references for reflectance conversion in a captured scene for potential ground-based hyperspectral image surveillance systems suffers from security, as it exposes the location of the system, and could be expensive for a broad application due to the high prices of standard Spectralons. In Part I of this two-part article, the state-of-the-art of ground-based hyperspectral imaging systems for explosive trace detection was discussed along with the future challenges for generic surveillance systems. The part II of the article proposes different approaches for reflection conversion to compensate the usage of white references at every capturing instant during surveillance, which is one of the main challenges of these systems. As an initial approach, recording the radiance of white reference only at the initialization of the system is tested by capturing an experimental setup in different days and times throughout a duration of two months. The correlations between the incident lights obtained from white references at different times, days, and orientations are compared. The effects of using a prerecorded white radiance for the reflectance conversion in the current scene are investigated. Second, a practical approach for reflectance calculation is tested by employing cheap-price reference objects having smooth characteristics in short-wave infrared spectrum, instead of expensive Spectralons. Finally, a novel reflectance conversion method is developed by using the reflectance of the static background materials in the scene rather than standard references by considering the stationary characteristics of surveillance systems. The proposed approaches are compared and verified on the baseline detection methods.

Ground-Based Hyperspectral Image Surveillance Systems for Explosive Detection: Part I—State of the Art and Challenges

Existing ground-based hyperspectral imaging (HSI) systems provide a good potential for standoff detection of explosives and their traces. The current technology, however, has still essential challenges to achieve a generic surveillance system for dynamic scenes with moving vehicles. In this regard, part I of this two-part article presents a state-of-the-art of existing HSI systems for explosive detection and discusses the future challenges for such a surveillance system. Considering the utilization of a light source and the operating spectral region, the presented overview classifies the related HSI systems as active and passive systems in the long-wave and short-wave infrared spectra, and investigates the methods in each class with respect to the targeted explosives, illumination and capturing devices, target detection algorithms, and performance evaluation methodologies. The investigation has revealed the major challenges for a generic surveillance system as 1) a thorough experimental performance validation with respect to time, date, and orientation, 2) sufficient acquisition speeds to capture moving vehicles, 3) registration and regulation of different spectral bands captured at different positions of the movement in a dynamic scene, 4) lower false positive rates required for the dynamic scenes compared to static control points, and 5) white reference compensations for the reflectance conversion in a continuous and secure surveillance system. The companion part II article then provides different solutions for reflection conversion to eliminate the dependencies to the white references for an economic and sustainable system.

A Hardware-Friendly Hyperspectral Lossy Compressor for Next-Generation Space-Grade Field Programmable Gate Arrays

The increment in the data rate of the new-generation hyperspectral sensors is making more critical the necessity of lossy compression solutions able to achieve higher compression ratios (CRs). In this paper, a transform-based lossy compressor, namely lossy compression algorithm for hyperspectral image systems (HW-HyperLCA), is proposed as a modification of the original HyperLCA, specifically designed for being executed into space-grade field programmable gate array (FPGA) devices. The HW-HyperLCA compressor can be totally executed using integer arithmetic, using different levels of precision that can be adapted for achieving the best relation between compression performance and computational burden. The experiments carried out in this paper demonstrate that the HW-HyperLCA compressor is able to compress the hyperspectral images to less than 1 bit per pixel per band with low distortions (more than 40 dB of signal-to-noise ratio) while the performed complexity analysis demonstrates that it can be efficiently executed into space-grade FPGAs using very few computational resources and good results in terms of maximum clock frequency can be obtained.

Automated Core Logging Technology for Geotechnical Assessment: A Study on Core from the Cadia East Porphyry Deposit

Abstract The Cadia East porphyry deposit, located approximately 20 km south of Orange, New South Wales, Australia, contains a significant resource of copper and gold. This resource is hosted within the Forest Reefs Volcanics and is spatially and temporally associated with the Cadia Intrusive Complex. To extract ore, the underground mine currently uses the block cave mining method. The Cadia East geotechnical model provides data inputs into a range of numerical and empirical analysis methods that make up the foundation for mine design. These data provide input into the construction of stress models, caveability models, ground support design, and fragmentation analysis. This geotechnical model encompasses two commonly used rock classification systems that quantify ground conditions: (1) rock mass rating (RMR) and (2) rock tunneling quality index (Q index). The RMR and Q index are calculated from estimates of rock quality designation (RQD), number of fracture sets, fracture roughness, fracture alteration, and fracture spacing. Geologists and geotechnical engineers collect information used to produce these estimates by manually logging sections of drill core, a time-consuming task that can result in inconsistent data. Modern automated core scanning technologies offer opportunities to rapidly collect data from larger samples of drill core. These automated core logging systems generate large volumes of spatially and spectrally consistent data, including a model of the drill core surface from a laser profiling system. Core surface models are used to extract detailed measurements of fracture location, orientation, and roughness from oriented drill core. These data are combined with other morphological and mineralogical outputs from automated hyperspectral core logging systems to estimate RMR and the Q index systematically over contiguous drill core intervals. The goal of this study was to develop a proof-of-concept methodology that extracts geotechnical index parameters from hyperspectral and laser topographic data collected from oriented drill core. Hyperspectral data from the Cadia East mine were used in this case study to assess the methods. The results show that both morphological and mineralogical parameters that contribute to the RMR and Q index can be extracted from the automated core logging data. This approach provides an opportunity to capture consistent geologic, mineralogical, and geotechnical data at a scale that is too time-consuming to achieve via manual data collection.

DeepCubeNet: reconstruction of spectrally compressive sensed hyperspectral images with deep neural networks.

Several hyperspectral (HS) systems based on compressive sensing (CS) theory have been presented to capture HS images with high accuracy and with a lower number of measurements than needed by conventional systems. However, the reconstruction of HS compressed measurements is time-consuming and commonly involves hyperparameter tuning per each scenario. In this paper, we introduce a Convolutional Neural Network (CNN) designed for the reconstruction of HS cubes captured with CS imagers based on spectral modulation. Our Deep Neural Network (DNN), dubbed DeepCubeNet, provides significant reduction in the reconstruction time compared to classical iterative methods. The performance of DeepCubeNet is investigated on simulated data, and we demonstrate for the first time, to the best of our knowledge, real reconstruction of CS HS measurements using DNN. We demonstrate significantly enhanced reconstruction accuracy compared to iterative CS reconstruction, as well as improvement in reconstruction time by many orders of magnitude.

Evaluation of peripheral artery disease with the TIVITA® Tissue hyperspectral imaging camera system.

Objective, reliable and easy screening for peripheral artery disease (PAD) is essential to confirm the diagnosis and initiate the respective treatment. Therefore, a new non-invasive hyperspectral camera (TIVITA® Tissue) was tested in patients with and without PAD. It was hypothesized that the oxygenation parameters of the TIVITA® Tissue correlate to established modalities for detection of PAD and allow differentiation between individuals with and without PAD. Evaluation of tissue oxygenation was performed in the angiosome of the medial plantar artery in 25 healthy young people and in 24 patients with and 25 patients without PAD in comparable age. Thereby, superficial oxygenation (StO2) and near-infrared (NIR) perfusion index were measured with the TIVITA® Tissue. Additionally, the ankle-brachial-index (ABI), the complaint free walking distance and the vascular quality of life were assessed and demographic data were obtained from all participants. TIVITA® Tissue analysis revealed significantly reduced StO2 and NIR perfusion index in PAD compared to healthy young participants and patients without PAD. StO2 and NIR perfusion index positively correlated with ABI, the complaint free walking distance and the vascular quality of life score. In summary, this new hyperspectral imaging camera bears great potential for PAD screening as well as for follow up.

Single pixel hyperspectral bioluminescence tomography based on compressive sensing.

Photonics based imaging is a widely utilised technique for the study of biological functions within pre-clinical studies. Specifically, bioluminescence imaging is a sensitive non-invasive and non-contact optical imaging technique that is able to detect distributed (biologically informative) visible and near-infrared activated light sources within tissue, providing information about tissue function. Compressive sensing (CS) is a method of signal processing that works on the basis that a signal or image can be compressed without important information being lost. This work describes the development of a CS based hyperspectral Bioluminescence imaging system that is used to collect compressed fluence data from the external surface of an animal model, due to an internal source, providing lower acquisition times, higher spectral content and potentially better tomographic source localisation. The work demonstrates that hyperspectral surface fluence images of both block and mouse shaped phantom due to internal light sources could be obtained at 30% of the time and measurements it would take to collect the data using conventional raster scanning methods. Using hyperspectral data, tomographic reconstruction of internal light sources can be carried out using any desired number of wavelengths and spectral bandwidth. Reconstructed images of internal light sources using four wavelengths as obtained through CS are presented showing a localisation error of ∼3 mm. Additionally, tomographic images of dual-colored sources demonstrating multi-wavelength light sources being recovered are presented further highlighting the benefits of the hyperspectral system for utilising multi-colored biomarker applications.

Spatial Distortion Assessments of a Low-Cost Laboratory and Field Hyperspectral Imaging System.

This article describes the adaptation of an existing aerial hyperspectral imaging system in a low-cost setup for collecting hyperspectral data in laboratory and field environment and spatial distortion assessments. The imaging spectrometer system consists of an ImSpector V9 hyperspectral pushbroom scanner, PixelFly high performance digital CCD camera, and a subsystem for navigation, position determination and orientation of the system in space, a sensor bracket and control system. The main objective of the paper is to present the system, with all its limitations, and a spatial calibration method. The results of spatial calibration and calculation of modulation transfer function (MTF) are reported along with examples of images collected and potential uses in agronomy. The distortion value rises drastically at the edges of the image in the near-infrared segment, while the results of MTF calculation showed that the image sharpness was equal for the bands from the visible part of the spectrum, and approached Nyquist’s theory of digitalization. In the near-infrared part of the spectrum, the MTF values showed a less sharp decrease in comparison with the visible part. Preliminary image acquisition indicates that this hyperspectral system has potential in agronomic applications.