Spectral Camera Research Articles

Detecting Water Stress in Winter Wheat Based on Multifeature Fusion from UAV Remote Sensing and Stacking Ensemble Learning Method

The integration of remote sensing technology and machine learning algorithms represents a new research direction for the rapid and large-scale detection of water stress in modern agricultural crops. However, in solving practical agricultural problems, single machine learning algorithms cannot fully explore the potential information within the data, lacking stability and accuracy. Stacking ensemble learning (SEL) can combine the advantages of multiple single machine learning algorithms to construct more stable predictive models. In this study, threshold values of stomatal conductance (gs) under different soil water stress indices (SWSIs) were proposed to assist managers in irrigation scheduling. In the present study, six irrigation treatments were established for winter wheat to simulate various soil moisture supply conditions. During the critical growth stages, gs was measured and the SWSI was calculated. A spectral camera mounted on an unmanned aerial vehicle (UAV) captured reflectance images in five bands, from which vegetation indices and texture information were extracted. The results indicated that gs at different growth stages of winter wheat was sensitive to soil moisture supply conditions. The correlation between the gs value and SWSI value was high (R2 > 0.79). Therefore, the gs value threshold can reflect the current soil water stress level. Compared with individual machine learning models, the SEL model exhibited higher prediction accuracy, with R2 increasing by 6.67–17.14%. Using a reserved test set, the SEL model demonstrated excellent performance in various evaluation metrics across different growth stages (R2: 0.69–0.87, RMSE: 0.04–0.08 mol m−2 s−1; NRMSE: 12.3–23.6%, MAE: 0.03–0.06 mol m−2 s−1) and exhibited excellent stability and accuracy. This research can play a significant role in achieving large-scale monitoring of crop growth status through UAV, enabling the real-time capture of winter wheat water deficit changes, and providing technical support for precision irrigation.

Achieving wheat seedling freezing injury assessment during the seedling stage using Unmanned Ground Vehicle (UGV) and hyperspectral imaging technology

Freezing injury may cause irreversible damage to wheat (Triticum aestivum L) tissues and can significantly reduce yield and quality. Therefore, quick and non-destructively estimating the degree of frost damage for formulating anti-freezing protection strategies and preventing frost damage is very crucial. In this study, we obtained hyperspectral images of wheat leaves for accurate identification of frost damage. A remote-controlled Unmanned Ground Vehicle (UGV) equipped with an imaging spectral camera was used to capture the hyperspectral images of frost-damaged wheat leaves. We compared the efficiency of two methods (the one without removal of weeds, and the other is to remove the corresponding area of weeds from the hyperspectral image by Deeplab V3+) for estimation of wheat freezing damage degree by using four different algorithms; Support Vector Machine Classification (SVM), Mahalanobis Distance Classification (MaD), Minimum Distance Classification (MiD), and Maximum Likelihood Classification (ML). We found that, Deeplab V3+ can efficiently identify the weeds from hyperspectral images, as the overall accuracy (OA) values of different algorithms were high in images with weeds removal as compared to the values in weeds containing images. Further, applying ML model after weeds removal have high OA (93.26 %) as compared to the other models. Thus, using Deeplab V3+ and ML can be a potential approach to identify the freezing injury in wheat for sustainable agricultural productivity.

Compact staring-type underwater spectral imaging system utilizing k-Nearest neighbor-based interpolation for spectral reconstruction

Besides the challenges posed by the complex optical properties of water, underwater spectral imaging face significant issues, including system bulkiness, optical complexity, and high spectral resolution requirement. This study introduces a compact staring-type underwater spectral imaging system designed to overcome these limitations. Featuring a 6-channel rotary spectral camera, the system is optimized for optical simplicity, and spectral efficiency. It employs a k-Nearest Neighbor (KNN)-based spectral reconstruction method to substantially enhance spectral resolution. Auto-focusing and image registration techniques, utilizing the Tenengrad function and Scale-Invariant Feature Transform (SIFT) feature points, ensure sharp and aligned multi-channel imagery. Experimental validation demonstrates the system’s capability to accurately reconstruct spectral reflectance data across the 400 nm to 700 nm band, significantly increasing spectral resolution from 6 to 31 channels. The spectral reflectance reconstruction results, achieving an average Root Mean Square Error (RMSE) of 0.0651 and Goodness of Fit Coefficient (GFC) of 0.9871, highlight the system’s efficiency in precise spectral analysis. This advancement in spectral resolution and adaptability to diverse underwater applications marks a significant leap forward in underwater observation and monitoring through spectral imaging.

Do spectral reflectance indices distinguish between the greenness in three different moss species in moss banks on Galindez Island (Argentine Islands)?

Spectral reflectance indices of green state of Warnstorfia fontinaliopsis, Chorisodontium aciphyllum and Sanionia georgicouncinata on moss bank in the Galindez Island (Argentine Islands) were measured using a handheld spectrometer PolyPen RP 410 UVIS (Photon Systems Instruments, Drásov, Czech Republic) within the range of 380–790 nm in order to find suitable ones for effective classification of moss species within the same colour state (green). Among altogether 19 indices tested, there were some which did not differ significantly between the studied species (subgroup 1). Other indices (subgroup 2) were sensitive enough to distinguish one of the studied species from the others, and finally (subgroup 3), they were found statistically significantly different for all studied moss species. Also, the indices calculated at wavelengths typical for UAV spectral cameras (green, red and red edge channels) showed species-specific differences and can be potentially used to distinguish between different mosses within the same green physiological state indicating a good vigor.

Detection of hypertension using a target spectral camera: a prospective clinical study

Hypertension is a significant contributor to premature mortality, and the regular monitoring of blood pressure (BP) enables the early detection of hypertension and cardiovascular disease. There is an urgent need for the development of highly accurate cuffless BP devices. We examined BP measurements based on a target spectral camera’s recordings and evaluated their accuracy. Images of 215 adults’ palms and faces were recorded, and BP was measured. The camera captured RGB wavelength data at 640 × 480 pixels and 150 frames per second (fps). These recordings were analyzed to extract pulse transit time (PTT) values between the face and palm, a key parameter for estimating BP. Continuous BP measurements were taken using a CNAPmonitor500 for validation. Three frequency wavelengths were measured from video images. A machine learning model was constructed to determine hypertension, defined as a systolic BP of 130 mmHg or higher or a diastolic BP of 80 mmHg or higher, using the visualized data. The discrimination between hypertension and normal BP was 95.0% accurate within 30 s and 90.3% within 5 s, based on the captured images. The results of heartbeat-by-heartbeat analyses can be used to determine hypertension based on only one second of camera footage or one heartbeat. The data extracted from a video recorded by a target spectral camera enabled accurate hypertension diagnoses, suggesting the potential for simplified BP monitoring.

Synchronising an IMX219 image sensor and AS7265x spectral sensor to make a novel low-cost spectral camera

A low-cost novel spectral camera able to be used for near infrared spectroscopy was made by using a Jetson Nano to synchronize a Sony IMX219 NOIR autofocus image sensor, an AMS AS7265x 18-channel spectral sensor and Osram SFH 4737 broadband infrared LED’s. Synchronizing an image sensor and spectral sensor augments a standard RGB image with light spectrum information; capturing the light distribution information normally lost in RGB image capture. Sutherland et al. [1] used this novel spectral camera to examine the dorsal surface of juvenile lobsters as a possible pre-moult detector. Having the image and spectrum in combination allowed the incomplete and unmineralized post-moult dorsal surface to be characterized with 86.7% accuracy for the first time. A proposed application for the spectral camera is to omit the local SFH 4737 light source and use the camera in daylight, effectively making a low-cost substitute hyperspectral snapshot camera. In this configuration the camera may have application for low-cost drone deployment for small scale agriculture.

Performance Characterization of an Illumination-Based Low-Cost Multispectral Camera.

Spectral imaging has many applications, from methane detection using satellites to disease detection on crops. However, spectral cameras remain a costly solution ranging from 10 thousand to 100 thousand euros for the hardware alone. Here, we present a low-cost multispectral camera (LC-MSC) with 64 LEDs in eight different colors and a monochrome camera with a hardware cost of 340 euros. Our prototype reproduces spectra accurately when compared to a reference spectrometer to within the spectral width of the LEDs used and the ±1σ variation over the surface of ceramic reference tiles. The mean absolute difference in reflectance is an overestimate of 0.03 for the LC-MSC as compared to a spectrometer, due to the spectral shape of the tiles. In environmental light levels of 0.5 W m-2 (bright artificial indoor lighting) our approach shows an increase in noise, but still faithfully reproduces discrete reflectance spectra over 400 nm-1000 nm. Our approach is limited in its application by LED bandwidth and availability of specific LED wavelengths. However, unlike with conventional spectral cameras, the pixel pitch of the camera itself is not limited, providing higher image resolution than typical high-end multi- and hyperspectral cameras. For sample conditions where LED illumination bands provide suitable spectral information, our LC-MSC is an interesting low-cost alternative approach to spectral imaging.

Sensing Pre- and Post-Ecdysis of Tropical Rock Lobsters, Panulirus ornatus, Using a Low-Cost Novel Spectral Camera

Tropical rock lobsters (Panulirus ornatus) are a highly cannibalistic species with intermoult animals predominantly attacking animals during ecdysis (moulting). Rapid, positive characterisation of pre-ecdysis lobsters may open a pathway to disrupt cannibalism. Ecdysial suture line development is considered for pre-ecdysis recognition with suture line definition compared for intermoult and pre-ecdysis lobsters emerged and immerged, using white, near ultraviolet (365 nm), near infrared (850 nm), and specialty SFH 4737 broadband IR LEDs against a reference of intermoult lobsters with no suture line development. Difficulties in acquiring suture line images prompted research into pre-ecdysis characterisation from the lobster’s dorsal carapace, due to its accessibility through a culture vessel’s surface. In this study, a novel low-cost spectral camera was developed by coordinating an IMX219 image sensor, an AS7265x spectral sensor, and four SFH 4737 broadband infrared LEDs through a single-board computer. Images and spectral data from the lobster’s dorsal carapace were acquired from intermoult, pre-ecdysis, and post-ecdysis lobsters. For the first time, suture line definition was found to be enhanced under 850 nm, 365 nm, and SFH 4737 LEDs for immerged lobsters, while the 850 nm LED achieved the best suture line definition of emerged lobsters. Although the spectral camera was unable to characterise pre-ecdysis, its development was validated when a least squares regression for binary classification decision boundary successfully separated 86.7% of post-ecdysis lobsters. Achieving post-ecdysis characterisation is the first time the dorsal carapace surface has been used to characterise a moult stage for palinurid lobsters.

High-precision spectra captured by a spectral camera and suppression of their nonlinearity

The high sensitivity of photoplethysmography (PPG) spectral signals provides conditions for extracting dynamic spectra carrying nonlinear information. By the idea of spatial conversion precision, this paper uses a spectral camera to collect highly sensitive spectral data of 24 wavelengths and proposes a method for extracting dynamic spectra of three different optical path lengths and their joint modeling. In the experiment, the models of the red blood cells and white blood cells established by the joint spectra achieved good results, with the correlation coefficients above 0.77. This study has great significance for achieving high-precision noninvasive quantitative analysis of human blood components.

Design of a Multimodal Detection System Tested on Tea Impurity Detection

A multimodal detection system with complementary capabilities for efficient detection was developed for impurity detection. The system consisted of a visible light camera, a multispectral camera, image correction and registration algorithms. It can obtain spectral features and color features at the same time and has higher spatial resolution than a single spectral camera. This system was applied to detect impurities in Pu’er tea to verify its high efficiency. The spectral and color features of each pixel in the images of Pu’er tea were obtained by this system and used for pixel classification. The experimental results showed that the accuracy of a support vector machine (SVM) model based on combined features was 93%, which was 7% higher than that based on spectral features only. By applying a median filtering algorithm and a contour detection algorithm to the label matrix extracted from pixel-classified images, except hair, eight impurities were detected successfully. Moreover, taking advantage of the high resolution of a visible light camera, small impurities could be clearly imaged. By comparing the segmented color image with the pixel-classified image, small impurities such as hair could be detected successfully. Finally, it was proved that the system could obtain multiple images to allow a more detailed and comprehensive understanding of the detected items and had an excellent ability to detect small impurities.

Internet of Things assisted Unmanned Aerial Vehicle for Pest Detection with Optimized Deep Learning Model

IoT technologies & UAVs are frequently utilized in ecological monitoring areas. Unmanned Aerial Vehicles (UAVs) & IoT in farming technology can evaluate crop disease & pest incidence from the ground’s micro & macro aspects, correspondingly. UAVs could capture images of farms using a spectral camera system, and these images are been used to examine the presence of agricultural pests and diseases. In this research work, a novel IoT- assisted UAV- based pest detection with Arithmetic Crossover based Black Widow Optimization-Convolutional Neural Network (ACBWO-CNN) model is developed in the field of agriculture. Cloud computing mechanism is used for monitoring and discovering the pest during crop production by using UAVs. The need for this method is to provide data centers, so there is a necessary amount of memory storage in addition to the processing of several images. Initially, the collected input image by the UAV is assumed on handling the via-IoT-cloud server, from which the pest identification takes place. The pest detection unit will be designed with three major phases: (a) background &foreground Segmentation, (b) Feature Extraction & (c) Classification. In the foreground and background Segmentation phase, the K-means clustering will be utilized for segmenting the pest images. From the segmented images, it extracts the features including Local Binary Pattern (LBP) &improved Local Vector Pattern (LVP) features. With these features, the optimized CNN classifier in the classification phase will be trained for the identification of pests in crops. Since the final detection outcome is from the Convolutional Neural Network (CNN); its weights are fine-tuned through the ACBWO approach. Thus, the output from optimized CNN will portray the type of pest identified in the field. This method’s performance is compared to other existing methods concerning a few measures.

Monitoring growth and predicting crop yield through UAV-mounted spectral camera analysis of the interplay between soil compaction and vegetation index

This paper aims to introduce a prediction of crop yield based on relationship between soil compaction and vegetation index. The soil compaction increasingly with depth, which was calculated manuallyunevenly distributed in the field. The NDVI/NDRE was conducted by aerial spectral images taken by the UAV. To figure out connection, the Pearson’s correlation test was applied to analyze the correlation between factors. These research results show that the NDVI/NDRE in WS and SA crops increased and decreased steadily after reaching the maximum values (0.85 ± 0.02/ 0.38 ± 0.02 and 0.8 ± 0.02/ 0.28 ± 0.02) during the reproductive stage. The NDVI/NDRE had a high relationship with the plant height, tiller number, yield components of rice. WS and SA networks were built and tested according to the training algorithm in the Matlab software for predicting rice yield with high reliability. The developed models showcase promising results in forecasting rice yield, underscoring the potential applicability of this methodology in agricultural yield prediction.

Balancing composite motion optimization using R-ERNN with plant disease

Soybean is a crop with a long cultivation history that plays a significant role on agricultural field. Early diagnosis of Soybean Mosaic Virus Disease (SMVD) is must require, because it causes a fast reduction and significant losses in soybean yields. Therefore, this paper proposes a Balancing Composite Motion optimization utilizing Recalling-Enhanced Recurrent Neural Network with Plant Disease for separating its severity into 0, 1 and 2 grades. Initially, hyper spectral image data was taken from Spec View software. Then, the spectral imageries of soybean leaves are processed to remove the redundancy in frequency bands that emerges owing to hyper spectral camera does not present any important information are filtered by Dual Tree Complex Wavelet Transform (DTCWT). In this, hyper spectral image features are extracted using Ternary pattern and discrete wavelet transform (TP-DWT) method. After completing the process of feature extraction, the feature extracted imageries are given to Recalling-Enhanced Recurrent Neural Network (RE-RNN) and it is used to classify the Soybean mosaic disease. Here, Balancing Composite Motion Optimization (BCMO) algorithm is employed for tuning the RE-RNN hyper parameters. The proposed PDI-RE-RNN-BCMO-HSI method provides 27.03%, 28.94% and 39.04% higher precision compared with existing method like grading model of soybean mosaic infection depending upon hyper spectral imaging technology (PDI-CNN-SVM-HSI), recognization of soybean varieties under hyper spectral imaging technology and one-dimensional CNN (PDI-1D-CNN-HSI) and hyper spectral imaging technology combined with multivariate models to recognize soybean disease (PDI-SNV-ELM-HSI) respectively.

Progressive Self-supervised Learning for CASSI Computational Spectral Cameras

Progressive Self-supervised Learning for CASSI Computational Spectral Cameras

HYPERSPECTRAL IMAGING IN PRESERVATION OF CROATIA’S HISTORIC TREES: A CASE STUDY OF DEDEK OAK IN MAKSIMIR PARK

Abstract. This study focuses on the application of hyperspectral imaging techniques for the genetic preservation of historic trees, specifically investigating the 600-year-old "Dedek" Oak in Maksimir Park, Croatia. The research aims to contribute to the conservation of Croatia's natural heritage and underscore the potential of hyperspectral imaging in tree preservation. The Croatian Forest Research Institute has been actively involved in preserving the pedunculate oak trees, recognizing their cultural and ecological significance. Efforts have been made to cultivate seedlings from the original genetic material, thereby preventing the loss of valuable genes and ensuring their long-term preservation. Spectral signatures of "Dedek" Oak and its clones were captured using high-performance hyperspectral push broom cameras, including the HySpex VNIR-1800 and SWIR-384. Spectral camera scanning was conducted in the field and laboratory. The comparative analysis of the spectral samples revealed unique characteristics and demonstrated the effectiveness of hyperspectral imaging in studying historic trees for preservation purposes. The spectral signatures of vegetation display dynamic characteristics in terms of spectral resolution. Collecting and documenting these signatures is considerably more challenging, and their integration into spectral libraries should be approached with careful consideration. There are several spectral libraries that are organized by chapters and consist of samples that have a sufficient number of analysis and documentation to determine the quality of the spectrum. In this study, spectral signatures of pedunculate oak (Quercus robur L.) called Dedek and its clones were singled out. The objective of this research was to establish a foundation for a spectral library that would facilitate future research on the application of hyperspectral scanners for detecting protected tree species and their clones, all for the purpose of preservation of the genetic diversity of protected trees in Croatia. The findings of this research contribute to the conservation of Croatia's natural heritage by providing valuable insights into the genetic preservation of the 600-year-old "Dedek" Oak. Also, determining the spectral signatures of clones can help us with the identification of specific tree species (especially during aerial imaging, which is planned to be conducted in the future). In conclusion, this research showcases the importance of preserving historic trees and the potential of hyperspectral imaging as a valuable tool for genetic preservation efforts. The study's outcomes contribute to the long-term conservation of Croatia's natural heritage and provide a foundation for future research in the field of tree preservation and spectral library inception.

Evaluation of land roughness and weather effects on paddy field using cameras mounted on drone: A comprehensive analysis from early to mid-growth stages

This paper proposes a potential solution to the traditional method of rice field preparation and management by utilizing unmanned aerial vehicles (UAVs) equipped with RGB and spectral cameras. Farmers typically rely on their experience to deal with bad weather and prepare rice fields in the beginning of the season, resulting in time-consuming and inefficient processes. The proposed method uses UAVs to evaluate the rice fields from the start to the middle stages of the season. The UAV flight follows a pre-determined trajectory autonomously, capturing successive images using the attached cameras. In the soil preparation stage, Harralick texture segmentation is used to analyze the roughness of paddy fields’ surfaces (low land, normal land and high land). In this approach, image is segmented first, then areas belonging the same clusters are extracted, manually verified and labelled. These labelled images are used as training data for the proposed model. The result shows that roughness of field surface can be observed clearly with high accuracy at over 90%, with random forest method is at over 97%. In addition to this, the effects of rain and flooding are also considered during the reproductive stages. The results show that the proposed method achieves high accuracy in evaluating rice fields, which can significantly increase efficiency and productivity compared to traditional methods. The study highlights the potential of UAV-based monitoring to provide a cost-effective and efficient way to manage rice fields and improve crop yields, leading to higher food security and sustainable agriculture.

Snapshot Multispectral Light‐Field Imaging Based on Nematic Liquid‐Crystal Microlens Array via Deep Learning

In this study, a multispectral light‐field imaging system is developed using nematic liquid‐crystal microlens arrays (LC‐MLAs). The system solves the spectral under‐sampling and complex system issues in existing light‐field and spectral cameras. The LC‐MLA is the core optical element of the snapshot imaging system. No dispersion or customized scanning machinery is required to obtain multispectral light fields. Deep learning technology is utilized to reconstruct spectral information from raw data. In the experiments, it is demonstrated that the proposed method efficiently reconstructs multispectral data with 10 nm accuracy in the visible wavelength range of 400–700 nm. The system offers high precision, simple equipment, and relatively low cost without additional dispersion and filtering devices.

SFA-guided mosaic transformer for tracking small objects in snapshot spectral imaging

This paper presents an end-to-end solution, the Spectral Filter Array (SFA)-guided Mosaic Transformer (SMT), designed for tracking small objects within mosaic spectral videos captured by snapshot spectral cameras. Tracking small objects amidst complex scenes poses greater challenges due to their variable appearances and limited feature representation. Spectral imaging, leveraging spectral and spatial information to characterize the material properties of objects, offers enhanced object feature discrimination compared to conventional visual imaging, making it an ideal choice for this task. Existing spectral tracking techniques, however, fall short in delivering satisfactory results for small objects due to their disruption of spatial-spectral aliasing correlations or disregard for small object characteristics. Hence, the proposed SMT leverages SFA guidance to model and aggregate multi-layer features effectively. Comprising the SFA-guided mosaic backbone (SMB), Multi-layer Feature Aggregation (MFA), and Prediction Head, SMT extracts hierarchical features directly from mosaic spectral images, amalgamates interdependencies between shallow-layer detail and deep-layer semantic information, and precisely locates small objects. Experiment results on our curated fully-annotated mosaic spectral small object tracking dataset, alongside a public normal-sized object tracking dataset, showcase SMT’s prowess in adeptly tracking small objects amidst challenging scenarios such as occlusion and drift. Specifically, our SMT achieves gains ranging from 0.3% to 5.3% in average precision rate and from 2.0% to 5.1% in average success rate over the second-ranked trackers across various challenging attributes. The dataset and code are available at: https://github.com/Chenlulu1993/SMT.

An introduction to artificial intelligence in machine vision for postharvest detection of disorders in horticultural products

Horticultural products may suffer from surface, subsurface, and internal disorders. These lower the market value and may lead to consumer dissatisfaction. Today’s postharvest industry has a high interest in implementing technologies that allow efficient sorting of disordered products to avoid food loss, and to improve market value and consumer appreciation. Inspection by machine vision using RGB imaging and more advanced techniques such as spectral cameras, X-ray and magnetic resonance imaging (MRI), is increasingly implemented to non-destructively analyze disorders in horticultural products. However, for objective, robust and reliable detection of symptoms of disorders that are often even not obvious to detect by human vision, powerful data analysis methods are essential. In recent years, image analysis by artificial intelligence (AI) has evolved from traditional machine learning (ML) to advanced deep learning (DL), which is reaching impressive detection abilities. The aim of this article is to introduce recent developments in AI for extracting information on postharvest disorders from complex image data obtained by modern imaging systems to the non-AI expert. The main progress in AI for postharvest applications is the increasing adoption of available convolutional neural networks (CNNs), usually developed and tested with large open source image datasets elsewhere, to postharvest image data. Challenges in the design of DL models are identified, such as the need for large quantities of (labeled) data, the model explainability, and generalizability. Although proposed innovations often result in excellent detection abilities, their usability in an industrial context should be further considered, where uncontrollable parameters may negatively impact the performance.

Coded aperture snapshot spectral imaging fundus camera

Spectral imaging holds great promise for the non-invasive diagnosis of retinal diseases. However, to acquire a spectral datacube, conventional spectral cameras require extensive scanning, leading to a prolonged acquisition. Therefore, they are inapplicable to retinal imaging because of the rapid eye movement. To address this problem, we built a coded aperture snapshot spectral imaging fundus camera, which captures a large-sized spectral datacube in a single exposure. Moreover, to reconstruct a high-resolution image, we developed a robust deep unfolding algorithm using a state-of-the-art spectral transformer in the denoising network. We demonstrated the performance of the system through various experiments, including imaging standard targets, utilizing an eye phantom, and conducting in vivo imaging of the human retina.