Pushbroom Hyperspectral Imager Research Articles

CoastalWQL: An Open-Source Tool for Drone-Based Mapping of Coastal Turbidity Using Push Broom Hyperspectral Imagery

Uncrewed-Aerial Vehicles (UAVs) and hyperspectral sensors are emerging as effective alternatives for monitoring water quality on-demand. However, image mosaicking for largely featureless coastal water surfaces or open seas has shown to be challenging. Another pertinent issue observed is the systematic image misalignment between adjacent flight lines due to the time delay between the UAV-borne sensor and the GNSS system. To overcome these challenges, this study introduces a workflow that entails a GPS-based image mosaicking method for push-broom hyperspectral images, together with a correction method to address the aforementioned systematic image misalignment. An open-source toolkit, CoastalWQL, was developed to facilitate the workflow, which includes essential pre-processing procedures for improving the image mosaic’s quality, such as radiometric correction, de-striping, sun glint correction, and object masking classification. For validation, UAV-based push-broom hyperspectral imaging surveys were conducted to monitor coastal turbidity in Singapore, and the implementation of CoastalWQL’s pre-processing workflow was evaluated at each step via turbidity retrieval. Overall, the results confirm that the image mosaicking of the push-broom hyperspectral imagery over featureless water surface using CoastalWQL with time delay correction enabled better localisation of the turbidity plume. Radiometric correction and de-striping were also found to be the most important pre-processing procedures, which improved turbidity prediction by 46.5%.

A Customisable Data Acquisition System for Open-Source Hyperspectral Imaging.

Hyperspectral imagers, or imaging spectrometers, are used in many remote sensing environmental studies in fields such as agriculture, forestry, geology, and hydrology. In recent years, compact hyperspectral imagers were developed using commercial-off-the-shelf components, but there are not yet any off-the-shelf data acquisition systems on the market to deploy them. The lack of a self-contained data acquisition system with navigation sensors is a challenge that needs to be overcome to successfully deploy these sensors on remote platforms such as drones and aircraft. Our work is the first successful attempt to deploy an entirely open-source system that is able to collect hyperspectral and navigation data concurrently for direct georeferencing. In this paper, we describe a low-cost, lightweight, and deployable data acquisition device for the open-source hyperspectral imager (OpenHSI). We utilised commercial-off-the-shelf hardware and open-source software to create a compact data acquisition device that can be easily transported and deployed. The device includes a microcontroller and a custom-designed PCB board to interface with ancillary sensors and a Raspberry Pi 4B/NVIDIA Jetson. We demonstrated our data acquisition system on a Matrice M600 drone at a beach in Sydney, Australia, collecting timestamped hyperspectral, navigation, and orientation data in parallel. Using the navigation and orientation data, the hyperspectral data were georeferenced. While the entire system including the pushbroom hyperspectral imager and housing weighed 735 g, it was designed to be easy to assemble and modify. This low-cost, customisable, deployable data acquisition system provides a cost-effective solution for the remote sensing of hyperspectral data for everyone.

Practical recommendations and limitations for pushbroom hyperspectral imaging of tree stems

In this short communication, we present a pilot study testing a new close-range sensing technology – a portable, pushbroom hyperspectral camera – in varying field conditions in forests. We evaluate how measurement conditions affect the in situ collection of stem bark spectra. In situ spectral libraries of woody elements are needed in, e.g., physically-based remote sensing applications, biodiversity mapping, and 3D vegetation modeling. Recent technological advancements bring portable and close-range capable sensors, such as small pushbroom imaging spectrometers, for consumer and research use. However, it is important to investigate the strengths and limitations of sensors utilizing pushbroom technology. Spectral measurements under forest canopies are challenging due to varying illumination conditions, which can have a significant effect on the quality of the data. We acquired hyperspectral reflectance images of Norway spruce (Picea abies (L.) Karst), Scots pine (Pinus sylvestris L.), and silver birch (Betula pendula Roth) stem bark directly in the forest. For each tree we collected reflectance images at 30-minute intervals throughout a day from a fixed view angle. The most significant change in the measured spectra occurred due to spatially varying irradiance between the white reference panel and the bark surface. The spatial variation of irradiance had the largest effect on data quality in visible and red-edge regions, and the smallest in near-infrared. In non-diffuse conditions, changes in irradiance were often unpredictable as clouds or canopy elements moved in and out of the direct solar beams. Diffuse overcast days with clouds can extend the time window for measurements, making it a practical choice for acquiring hyperspectral images of stem bark. We concluded that with a well-planned measurement set-up it is possible to improve the precision of in situ collected spectra of stem bark.

Fast and effective classification of plastic waste by pushbroom hyperspectral sensor coupled with hierarchical modelling and variable selection

Plastic waste management represents a global challenge in the framework of sustainable production and consumption of resources. One of the most critical issues in plastic recycling is polymer separation, necessary to obtain high-quality secondary raw material flow streams. The aim of this work was to build a classification strategy, based on pushbroom hyperspectral imaging, able to recognize the most common polymers found in mixed plastic waste to be applied at recycling plant scale. After exploring polymer spectral differences by principal component analysis, a hierarchical partial least squares-discriminant analysis, based on the acquired full spectra, and a hierarchical interval partial least squares-discriminant analysis, based on selected variables, were tested and their performances were evaluated and compared. High quality classification results were obtained in both cases, demonstrating that the developed multi-class models can be utilized in a flexible way for quality control and/or for on-line sorting actions in recycling plants.

Application of UAV Push-Broom Hyperspectral Images in Water Quality Assessments for Inland Water Protection: A Case Study of Zhang Wei Xin River in Dezhou Distinct, China

A water quality parameter retrieval scheme based on the UAV push-broom hyperspectral images was designed and validated for assessing the ecological health of Zhang Wei Xin River in Dezhou distinct, China. First, a UAV carrying a push-broom hyperspectral imager that is lightweight and has a small size was used to acquire high spatial and hyperspectral resolution images. Then, the mosaicked reflectance data of the whole river were produced by a seamless image mosaicking method with high geometrical accuracy and spectral fidelity. Next, the in-field measurements of different parameters and the corresponding spectral reflectance from the mosaicked images at the sampling points were used to build the water quality parameter retrieval models for total phosphorus (TP), chlorophyll a (Chla), and total suspended solids (TSS). To validate the model, the retrieval results of the testing sampling points were compared with the measured parameters. The coefficients of determination R2 of TP, Chla, and TSS were 0.886, 0.918, and 0.968, respectively. The retrieved TP, Chla, and TSS maps showed that the water pollution of Zhang Wei Xin River is serious, the total phosphorus exceeds the standard, and the water body is in a state of eutrophication. The UAV-based hyperspectral remote sensing technique provides a cost-effective method for inland water monitoring at a local scale with high accuracy.

A Vicarious Technique for Understanding and Diagnosing Hyperspectral Spatial Misregistration.

Pushbroom hyperspectral imaging (HSI) systems intrinsically measure our surroundings by leveraging 1D spatial imaging, where each pixel contains a unique spectrum of the observed materials. Spatial misregistration is an important property of HSI systems because it defines the spectral integrity of spatial pixels and requires characterization. The IEEE P4001 Standards Association committee has defined laboratory-based methods to test the ultimate limit of HSI systems but negates any impacts from mounting and flying the instruments on airborne platforms such as unmanned aerial vehicles (UAV's) or drones. Our study was designed to demonstrate a novel vicarious technique using convex mirrors to bridge the gap between laboratory and field-based HSI performance testing with a focus on extracting hyperspectral spatial misregistration. A fast and simple extraction technique is proposed for estimating the sampled Point Spread Function's width, along with keystone, as a function of wavelength for understanding the key contributors to hyperspectral spatial misregistration. With the ease of deploying convex mirrors, off-axis spatial misregistration is assessed and compared with on-axis behavior, where the best performance is often observed. In addition, convex mirrors provide an easy methodology to exploit ortho-rectification errors related to fixed pushbroom HSI systems, which we will show. The techniques discussed in this study are not limited to drone-based systems but can be easily applied to other airborne or satellite-based systems.

Geometric Correction of BaySpec OCI-F Pushbroom Hyperspectral Images via the Sequential Geometric Mappings among Simultaneous Video Frames Estimated by Least Square Matching

ضرورت دسترسی به کاربردهای وسیع تصاویر ابرطیفی سبب توسعة سیستم‌های تصویربرداری نوآورانه و اقتصادی در ثبت این تصاویر شده است. به‌منظور استفاده از این تصاویر، لازم است ارتباط هندسی دقیقی میان آنها و فضای زمین برقرار شود و این فرایند نیازمند نقاط کنترلی بسیاری است. این نکته ضرورت توسعة راهکارهای اصلاح هندسی منطبق با ساختار هریک از این دوربین‌ها را بارز می‌کند. سنجندة (nm 400-1000) BaySpec OCI-F یکی از سیستم‌های نوآورانه‌ای است که تصاویر ابرطیفی را با هندسة تصویربرداری پوش‌بروم دریافت می‌کند. این سنجنده، علاوه‌بر یک سنسور پوش‌بروم، از یک سنسور فریم نیز بهره می‌برد که هم‌زمان با سنسور پوش‌بروم و با رزولوشن مکانی زمانی مشابه، تصویر را دریافت می‌کند. در این مقاله، روشی برای اصلاح هندسی تصاویر پوش‌برومِ این سنجنده بیان شده است. در بخش اول این روش، با توجه به ساختار تصویربرداری دوربین، ارتباط هندسی میان آرایة خطی و سنسور فریم در قالب پارامترهای کالیبراسیونی مشخص می‌شود. در ادامه، به‌کمک برآورد ارتباط هندسی میان تصاویر فریم متوالی، پیکسل‌های تصویر پوش‌بروم در کنار یکدیگر چیده و تصویر اصلاح‌شده تولید می‌شود. در این روش، ارتباط هندسی میان هر جفت فریم متوالی به‌طور مستقیم، ازطریق تناظریابی کمترین مربعات، محاسبه می‌شود. نتایج به‌دست‌آمده نشان می‌دهد که این روش، به‌طور متوسط، 2/62% از اعوجاجات هندسی تصویر خام را کاهش داده است. این کاهش سبب شده است متوسط دقت مدل‌های درون‌یاب عمومی سادة دوبعدی و سه‌بعدی بین فضای تصویر و زمین، به‌ترتیب، 9/39% و 1/34% افزایش یابد.

A Target-Based Non-Uniformity Self-Correction Method for Infrared Push-Broom Hyperspectral Sensors

Non-uniformity in the response of spectral image elements is an inevitable phenomenon in hyperspectral imaging, which mainly manifests itself as the presence of band noise in the acquired hyperspectral data. This problem is prominent in the infrared band owing to the detector material, operating environment, and other factors. Non-uniformity is an important factor that can affect the quality of the hyperspectral data, which has a serious impact on both data analysis and applications and requires corrections via technical means wherever possible. This paper proposes a novel target-based non-uniformity self-correction method for infrared push-broom hyperspectral images. The Mars Mineralogical Spectrometer (MMS) onboard the Tianwen-1 orbiter was used as the research and application object. The model is constructed and applied to the target scene characteristics and detection patterns of Mars remote sensing exploration, which are combined with the causes of noise generation in the infrared spectral image bands. The design of the MMS dual-channel Visible-Near-Infrared (V-NIR) and Near-Mid-Infrared (N-MIR) co-field of view co-target detection and laboratory calibration data for the V-NIR spectral band can achieve non-uniformity corrections (NUCs). Therefore, for the MMS in-orbit Mars exploration mission, the method selected spectral data (920–1055 nm) characterized by a reduced atmospheric influence to iteratively obtain the homogeneous region, which was used to calculate the non-uniformity correction factor for the N-MIR spectral band. This method was compared, validated, and evaluated with other conventional methods using both laboratory and in-orbit hyperspectral data. The results showed that the experimental data corrections were comparable to laboratory calibrations, with a maximum relative deviation of <2.6%. These results prove that our method not only provides an excellent non-uniformity correction, but also ensures spectral fidelity. It can thus be used as a non-uniformity correction process for the MMS and similar hyperspectral imagers.

Linear Spatial Misregistration Detection and Correction Based on Spectral Unmixing for FAHI Hyperspectral Imagery.

In push-broom hyperspectral imaging systems, the sensor rotation to the optical plane leads to linear spatial misregistration (LSM) in hyperspectral images (HSIs). To compensate for hardware defects through software, this paper develops four methods to detect LSM in HSIs. Different from traditional methods for grayscale images, the method of fitting the sum of abundance (FSAM) and the method of searching for equal abundance (SEAM) are achieved by hyperspectral unmixing for a selected rectangular transition areas containing an edge, which makes good use of spatial and spectral information. The method based on line detection for band-interleaved-by-line (BIL) images (LDBM) and the method based on the Fourier transform of BIL images (FTBM) aim to characterize the slope of line structure in BIL images and get rid of the dependence on scene and wavelength. A full strategy is detailed from aspects of data selection, LSM detection, and image correction. The full spectrum airborne hyperspectral imager (FAHI) is China's new generation push-broom scanner. The HSIs obtained by FAHI are tested and analyzed. Experiments on simulation data compare the four proposed methods with traditional methods and prove that FSAM outperforms other methods in terms of accuracy and stability. In experiments on real data, the application of the full strategy on FAHI verifies its effectiveness. This work not only provides reference for other push-broom imagers with similar problems, but also helps to reduce the requirement for hardware calibration.

Push-broom compressive hyperspectral imaging method based on self-fusion refinement

Compressive hyperspectral imaging (CHI) with random encoding mask usually suffers from various noises and artifacts. Inspired by the dual-camera CHI techniques based on hyperspectral (HS) and multispectral (MS) image fusion, herein, we present a single-camera push-broom CHI method based on self-fusion refinement (SFR). In this work, the MS guidance image for data fusion is derived directly from the raw solved HS data cube itself rather than any additional data source, which turns cross-fusion into self-fusion; furthermore, a modified joint bilateral filtering (JBF) fusion algorithm is developed to adapt this self-fusion problem, and an adaptive range Gaussian radius is adopted to avoid the invalidation or over-smoothing effects so as to ensure spatial and spectral improvement. The visualized and quantitative assessment results both demonstrate that the proposed method achieves high-quality HS imaging in terms of noise and artifact removal and spatial–spectral fidelity. Furthermore, the proposed method has a great flexibility and extensibility, whose performances highly depend on the exact fusion algorithm adopted, and a more suitable fusion algorithm will lead to better reconstruction quality; herein, the SFR process by the modified JBF achieves better performances than SFR by guided filtering (GF) or Markov random field (MRF).

HyScreen: A Ground-Based Imaging System for High-Resolution Red and Far-Red Solar-Induced Chlorophyll Fluorescence

Solar-induced chlorophyll fluorescence (SIF) is used as a proxy of photosynthetic efficiency. However, interpreting top-of-canopy (TOC) SIF in relation to photosynthesis remains challenging due to the distortion introduced by the canopy's structural effects (i.e., fluorescence re-absorption, sunlit-shaded leaves, etc.) and sun-canopy-sensor geometry (i.e., direct radiation infilling). Therefore, ground-based, high-spatial-resolution data sets are needed to characterize the described effects and to be able to downscale TOC SIF to the leafs where the photosynthetic processes are taking place. We herein introduce HyScreen, a ground-based push-broom hyperspectral imaging system designed to measure red (F687) and far-red (F760) SIF and vegetation indices from TOC with single-leaf spatial resolution. This paper presents measurement protocols, the data processing chain and a case study of SIF retrieval. Raw data from two imaging sensors were processed to top-of-canopy radiance by dark-current correction, radiometric calibration, and empirical line correction. In the next step, the improved Fraunhofer line descrimination (iFLD) and spectral-fitting method (SFM) were used for SIF retrieval, and vegetation indices were calculated. With the developed protocol and data processing chain, we estimated a signal-to-noise ratio (SNR) between 50 and 200 from reference panels with reflectance from 5% to 95% and noise equivalent radiance (NER) of 0.04 (5%) to 0.18 (95%) mW m-2 sr-1 nm-1. The results from the case study showed that non-vegetation targets had SIF values close to 0 mW m-2 sr-1 nm-1, whereas vegetation targets had a mean F687 of 1.13 and F760 of 1.96 mW m-2 sr-1 nm-1 from the SFM method. HyScreen showed good performance for SIF retrievals at both F687 and F760; nevertheless, we recommend further adaptations to correct for the effects of noise, varying illumination and sensor optics. In conclusion, due to its high spatial resolution, Hyscreen is a promising tool for investigating the relationship between leafs and TOC SIF as well as their relationship with plants' photosynthetic capacity.

Integrated fiber optic spectrally resolved downwelling irradiance sensor for pushbroom spectrometers.

We present an integrated fiber optic spectrally resolved downwelling irradiance sensor for pushbroom hyperspectral imagers. The system comprises of a cosine corrector and custom fiber patch cables, collecting the ambient light in a large solid angle and feeding it directly to the entrance slit of the spectrometer. The system enables simultaneous measurement of downwelling and upwelling irradiance using the main hyperspectral camera sensor. As a demonstration, the spectral reflectance of a soil sample was measured with a RMSE of 8.4%, a significant improvement on the RMSE of 54% found without correction. At a weight of approximately 10 grams, this system provides a substantial weight saving over standalone incident light sensing instruments.

Design and Dispersion Calibration of Direct-Vision Push-Broom Compressive Double-Amici-Prism Hyperspectral Imager

The design and calibration of the dispersive device in a hyperspectral imager significantly affect the performance of hyperspectral imaging, especially the spectral accuracy. To achieve high-accuracy hyperspectral imaging over the visible band, firstly, the geometric and dispersive parameters of the double Amici prism (DAP) that serves as a dispersive device in the direct-vision push-broom compressive hyperspectral imager (PBCHI) are designed and optimized; secondly, a calibration method based on the numerical calculation of the DAP model is put forward, which can turn the conventional pixel-wise dispersive shift calibration by a monochromator into a group of numerical calculations; lastly, a PBCHI prototype is built to test the performances of the designed and calibrated DAP and the hyperspectral imager. The calibration experiments demonstrate that the mean squared error (MSE) of the dispersive pixel shifts calibrated by the proposed numerical method is 0.1774, which indicates the calibration result of the proposed method is consistent with the directly calibrated result. Furthermore, after this numerical calculation, the spectral signatures of the reconstructed cubes of the DAP-based PBCHI system show consistency with the ground truth. This work will benefit the design and calibration of the DAP-based hyperspectral imager.

Burn characterization using object-oriented hyperspectral image classification.

This paper presents a new approach based on hyperspectral imaging combined with an object-oriented classification method that allows the generation of burn depth classification maps facilitating easier characterization of burns. Hyperspectral images of 14 patients diagnosed with burns on the upper and lower limbs were acquired using a pushbroom hyperspectral imaging system. The images were analyzed using an object-oriented classification approach that uses objects with specific spectral, textural and spatial attributes as the minimum unit for classifying information. The method performance was evaluated in terms of overall accuracy, sensitivity, precision and specificity computed from the confusion matrix. The results revealed that the approach proposed in this study performed well in differentiating burn classes with a high level of overall accuracy (95.99% ± 0.60%), precision (97.30% ± 2.46%), sensitivity (97.23% ± 3.02%) and specificity (98.02% ± 1.98%). In conclusion, the object-based approach for burns hyperspectral images classification can provide maps that can help surgeons identify with better precision different depths of burn wounds.

Hyperspectral imaging to measure apricot attributes during storage

The fruit industry needs rapid and non-destructive techniques to evaluate the quality of the products in the field and during the post-harvest phase. The soluble solids content (SSC), in terms of °Brix, and the flesh firmness (FF) are typical parameters used to measure fruit quality and maturity state. Hyperspectral imaging (HSI) is a powerful technique that combines image analysis and infrared spectroscopy. This study aimed to evaluate the potential of the application of the Vis/NIR push-broom hyperspectral imaging (400 to 1000 nm) to predict the firmness and the °Brix in apricots (180 samples) during storage (11 days). Partial least squares (PLS) and artificial neural networks (ANN) were used to develop predictive models. For the PLS, R2 values (test set) up to 0.85 (RMSEP=1.64 N) and 0.72 (RMSEP=0.51 °Brix) were obtained for the FF and SSC, respectively. Concerning the ANN, the best results in the test set were achieved for the FF (R2=0.85, RMSEP=1.50 N). The study showed the potential of the HSI technique as a non-destructive tool for measuring apricot quality even along the whole supply chain.

Generation of hyperspectral point clouds: Mapping, compression and rendering

Hyperspectral data are being increasingly used for the characterization and understanding of real-world scenarios. In this field, UAV-based sensors bring the opportunity to collect multiple samples from different viewpoints. Thus, light-material interactions of real objects may be observed in outdoor scenarios with a significant spatial resolution (5 cm/pixel). Nevertheless, the generation of hyperspectral 3D data still poses challenges in post-processing due to the high geometric deformation of images. Most of the current solutions use both LiDAR (Light Detection and Ranging) and hyperspectral sensors, which are integrated into the same acquisition system. However, these present several limitations due to errors derived from inertial measurements for data fusion and the spatial resolution according to the LiDAR capabilities. In this work, a method is proposed for the generation of hyperspectral point clouds. Input data are formed by push-broom hyperspectral images and 3D point clouds. On the one hand, the point clouds may be obtained by applying a typical photogrammetric workflow or LiDAR technology. Then, hyperspectral images are geometrically corrected and aligned with the RGB orthomosaic. Accordingly, hyperspectral data are ready to be mapped on the 3D point cloud. This procedure is implemented on the GPU by testing which points are visible for each pixel of the hyperspectral imagery. This work also provides a novel solution to generate, compress and render 3D hyperspectral point clouds, enabling the study of geometry and the hyperspectral response of natural and artificial materials in the real world.

OpenHSI: A Complete Open-Source Hyperspectral Imaging Solution for Everyone

OpenHSI is an initiative to lower the barriers of entry and bring compact pushbroom hyperspectral imaging spectrometers to a wider audience. We present an open-source optical design that can be replicated with readily available commercial-off-the-shelf components, and an open-source software platform openhsi that simplifies the process of capturing calibrated hyperspectral datacubes. Some of the features that the software stack provides include: an ISO 19115-2 metadata editor, wavelength calibration, a fast smile correction method, radiance conversion, atmospheric correction using 6SV (an open-source radiative transfer code), and empirical line calibration. A pipeline was developed to customise the desired processing and make openhsi practical for real-time use. We used the OpenHSI optical design and software stack successfully in the field and verified the performance using calibration tarpaulins. By providing all the tools needed to collect documented hyperspectral datasets, our work empowers practitioners who may not have the financial or technical capability to operate commercial hyperspectral imagers, and opens the door for applications in new problem domains.

Correction of Dropped Frames in High-resolution Push-broom Hyperspectral Images for Cultural Heritage

Dropped frames can occur in line-scan cameras, which result in non-uniform spatial sampling of the scene. A dropped frame occurs when data from an image sensor is not successfully recorded. When mosaicking multiple line-scan images, such as in high-resolution imaging, this can cause misalignment. Much previous work to identify dropped frames in video prioritises fast computation over high accuracy, whereas in heritage imaging, high accuracy is often preferred over short computation time. Two approaches to identify the position of dropped frames are presented, both using the A* search algorithm to correct dropped frames. One method aligns overlapping sections of push-broom images and the other aligns the push-broom image to a lower resolution reference image. The two methods are compared across a range of test images, and the method aligning overlapping sections is shown to perform better than the method using a reference image under most circumstances. The overlap method was applied to hyperspectral images acquired of La Ghirlandata, an 1873 oil on canvas painting by D. G. Rossetti, enabling a high-resolution hyperspectral image mosaic to be produced. The resulting composite image is 10,875 \( \times \) 14,697 pixels each with 500 spectral bands from 400–2,500 nm. This corresponds to a spatial resolution of \( 80 \,\mathrm{\upmu }\mathrm{m} \) and a spectral resolution of 3–6 nm.

Do-it-yourself VIS/NIR pushbroom hyperspectral imager with C-mount optics

This paper describes a new optomechanical design based on a previously presented do-it-yourself pushbroom hyperspectral imager (HSI) using commercial off-the-shelf (COTS) components. The new design uses larger aperture C-mount at F/2.8 instead of S-mount optics at F/4 to increase the throughput, which allows imaging at lower light levels. This is especially useful for dark surfaces like the deep ocean. The improved throughput is 6.77 higher at the center wavelength of 600 nm, which is shown both by theoretical calculations and experimental data. The measured full width at half maximum (FWHM) at 546.1 nm is 3.69 nm, which is close to the theoretical value of 3.3 nm, and smile and keystone are shown to be reduced in the new design. A method to characterize and remove second order effects using a cut-off filter is also presented and discussed.

Design and Experiment of Push-Broom Hyperspectral Microscopic Imaging System

Design and Experiment of Push-Broom Hyperspectral Microscopic Imaging System