Multispectral Imaging Technology Research Articles

Dual-Channel Mapping–Gas Column Concentration Inversion Method Based on Multispectral Imaging

Infrared multispectral imaging technology can achieve the long-distance, wide-ranging and fast detection of target gas, and has been widely used in the fields of dangerous-gas detection and environmental monitoring. However, due to the difficulty in acquiring background radiation as well as atmospheric disturbance and noise interference in the detection process, the quantitative detection of gas concentration has become a difficult problem to solve. Therefore, this paper proposes an inversion method for gas column concentration based on infrared multispectral imaging technology. Firstly, infrared background radiation images of the non-target gas absorption spectrum band were collected and converted into background radiation images of the target gas absorption spectrum band according to the dual-channel mapping relationship. Then, combined with the gas radiation images of the target gas absorption spectrum band, the column concentration distribution of the gas was obtained by using the measured calibration relationship between absorbance and column concentration. Experiments of gas detection in different environments were carried out, and the column concentration distribution of the target gas was inverted using this method; the results showed that the average relative error of the inversion of the gas column concentration was 4.84%, which enables the quantitative detection of gas column concentration in a complex environment.

High-performance plasmonic mid-infrared bandpass filters by inverse design.

Plasmonic spectral filters composed of periodic nanostructured metal films offer novel opportunities for the development of multispectral imaging technologies in the mid-infrared region. However, traditional plasmonic filters, which typically feature simplistic structures such as nanoholes or nanorings, are constrained by a narrow bandpass and significant crosstalk, leading to limited practical performance. Filters designed using inverse techniques allow a substantial degree of freedom in creating intricate structures that align with desired spectral characteristics, including a quasi-square spectral profile, high transmission, wide full width at half maximum, and reduced crosstalk. In this study, we have utilized an inverse design algorithm to engineer high-performance bandpass filters for the mid-infrared range, achieving an average transmittance exceeding 80% within the bandpass window and below 10% in the stop band, which is comparable to that of commercial multilayer Bragg filters. Nanofabrication processes were employed to transfer the designed pattern into the gold film on ZnS substrate that is transparent in the mid-infrared range. The resulting filters exhibit spectral performance analogous to that of the inversely designed models, making them suitable for direct integration with mid-infrared photodetector arrays in multispectral imaging systems.

A Snapshot Multi-Spectral Demosaicing Method for Multi-Spectral Filter Array Images Based on Channel Attention Network

Multi-spectral imaging technologies have made great progress in the past few decades. The development of snapshot cameras equipped with a specific multi-spectral filter array (MSFA) allow dynamic scenes to be captured on a miniaturized platform across multiple spectral bands, opening up extensive applications in quantitative and visualized analysis. However, a snapshot camera based on MSFA captures a single band per pixel; thus, the other spectral band components of pixels are all missed. The raw images, which are captured by snapshot multi-spectral imaging systems, require a reconstruction procedure called demosaicing to estimate a fully defined multi-spectral image (MSI). With increasing spectral bands, the challenge of demosaicing becomes more difficult. Furthermore, the existing demosaicing methods will produce adverse artifacts and aliasing because of the adverse effects of spatial interpolation and the inadequacy of the number of layers in the network structure. In this paper, a novel multi-spectral demosaicing method based on a deep convolution neural network (CNN) is proposed for the reconstruction of full-resolution multi-spectral images from raw MSFA-based spectral mosaic images. The CNN is integrated with the channel attention mechanism to protect important channel features. We verify the merits of the proposed method using 5 × 5 raw mosaic images on synthetic as well as real-world data. The experimental results show that the proposed method outperforms the existing demosaicing methods in terms of spatial details and spectral fidelity.

Evaluation of Dicamba Drift Injury and Yield Loss on Soybean Using Small Unmanned Aircraft Systems (sUAS) and Multispectral Imaging Technologies

Highlights The response of dicamba-susceptible soybean to three simulated rates and growth stages has been studied. Soybean physiological data, such as plant height and phototoxicity, was collected at the selected growth stage. Aerial images of the experiment field using UAV were collected on specific days after treatment. Soybean yield, biological response, and the correlation of vegetative indices in predicting soybean yield were achieved. Abstract. Many dicotyledonous weeds can be selectively controlled by dicamba application during the growing season. However, vapor or spray drift from dicamba application could cause significant yield loss on non-target species such as soybean [Glycine max (L.) Merrill] that do not contain the dicamba resistance trait. In this study, the impact of simulated dicamba drift was investigated on dicamba-susceptible soybeans during different growth stages. A field experiment was conducted employing small unmanned aircraft systems (sUAS) and multispectral image sensors to assess the dose-response relationship between dicamba drift and soybean yield. The experiment followed a randomized complete block design, with main plots representing three distinct soybean growth stages (third trifoliate, sixth trifoliate, and early reproductive stage) and sub-plots encompassing various simulated dicamba drift rates. The analysis, involving non-linear regression and normalized difference vegetation index (NDVI) regression, revealed that visible soybean injuries were observed at minimal dicamba rates. However, significant yield loss occurred during the V6 growth stage at a dicamba rate of 98 g.ai ha-1, emphasizing the critical growth stage sensitivity. Moreover, a moderate yet meaningful relationship (R2 = 0.46) was established between sUAS multispectral NDVI data and soybean yield, highlighting the potential for sUAS with multispectral sensors to predict soybean yield losses. These results contribute valuable insights into the impact of dicamba drift on soybeans and the efficacy of remote sensing technology in assessing yield outcomes. Keywords: Dicamba-drift injury, Multispectral image, Precision agriculture, Soybean yield, sUAS, Yield prediction.

Deep Learning Design for Multiwavelength Infrared Image Sensors Based on Dielectric Freeform Metasurface

AbstractNear‐infrared multispectral imaging technology enhances target detection and recognition by distinguishing the spectral characteristics of various targets. However, traditional imaging systems heavily rely on complex optical filter designs that are often bulky and mechanically unstable, posing significant challenges for miniaturization and integration challenging. In this study, a freeform dielectric metasurface with the wavelength‐multiplexing focusing effect based on a deep learning model is designed, which can separate the mixed near‐infrared light into distinct wavelengths. To effectively modulate the complex amplitude of the transmitted light at three distinct near‐infrared wavelengths (1150, 1350, and 1550 nm), high‐index silicon freeform nanostructures supporting rich resonant modes are proposed. An inverse design model based on deep learning is utilized to generate individual freeform nanostructures pixel by pixel, satisfying the complex amplitude requirement for a multiplexed metalens design. Both the simulated and experimental results show that the wavelength‐multiplexing effect of the devices is in good agreement with the target with negligible crosstalk. Finally, a metasurface is employed to realize near‐infrared multispectral imaging, which allows for the distinct detection and decoding of images at the three target wavelengths. The proposed technology has a wide range of applications in clinical medicine, biological tissue imaging, and deep‐space exploration.

Medición del marmoleado de carne de res usando imágenes espectrales: Un enfoque de regresión lineal múltiple

This study aimed at measuring beef marbling scores in an objective and simple manner through spectral imaging and multiple linear regression (MLR). Beef marbling prediction by hyperspectral imaging and partial least squares regression (PLSR) was analyzed to calibrate and evaluate an MLR model with a few selected wavelengths. Data came from 44 beef samples and consisted of their spectral signatures (75 wavelengths) from hyperspectral reflectance images (400-1000 nm) and their marbling scores assigned by evaluators. The wavelengths that presented regression coefficients with the highest absolute values in the PLSR model, were used to calibrate the MLR model by a backward stepwise approach (p < 0.05). The coefficient of determination for prediction (R2p) and the standard error of prediction (SEP) were evaluated. The MLR model was suitable for practical use because it required only 12 wavelengths for reliable predictions (R2p = 0.824 > 0.8; SEP = 11.4% < 15%). A model is proposed for the objective and simple measurement of beef marbling score using multispectral imaging technology.

Application of Multispectral Imaging Technology in Food Quality and Safety Inspection

Application of Multispectral Imaging Technology in Food Quality and Safety Inspection

Fluorescence-guided surgical system using holographic display: from phantom studies to canine patients.

Holographic display technology is a promising area of research that can lead to significant advancements in cancer surgery. We present the benefits of combining bioinspired multispectral imaging technology with holographic goggles for fluorescence-guided cancer surgery. Through a series of experiments with 43D-printed phantoms, small animal models of cancer, and surgeries on canine patients with head and neck cancer, we showcase the advantages of this holistic approach. The aim of our study is to demonstrate the feasibility and potential benefits of utilizing holographic display for fluorescence-guided surgery through a series of experiments involving 3D-printed phantoms and canine patients with head and neck cancer. We explore the integration of a bioinspired camera with a mixed reality headset to project fluorescent images as holograms onto a see-through display, and we demonstrate the potential benefits of this technology through benchtop and in vivo animal studies. Our complete imaging and holographic display system showcased improved delineation of fluorescent targets in phantoms compared with the 2D monitor display approach and easy integration into the veterinarian surgical workflow. Based on our findings, it is evident that our comprehensive approach, which combines a bioinspired multispectral imaging sensor with holographic goggles, holds promise in enhancing the presentation of fluorescent information to surgeons during intraoperative scenarios while minimizing disruptions.

Citrus Canopy SPAD Prediction under Bordeaux Solution Coverage Based on Texture- and Spectral-Information Fusion

Rapid and nondestructive prediction of chlorophyll content and response to the growth of various crops using remote sensing technology is a prominent topic in agricultural remote sensing research. Bordeaux mixture has been extensively employed for managing citrus diseases, such as black star and ulcer disease. However, the presence of pesticide residues in Bordeaux mixture can significantly modify the spectral response of the citrus canopy, thereby exerting a substantial influence on the accurate prediction of agronomic indices in fruit trees. In this study, we used unmanned aerial vehicle (UAV) multispectral imaging technology to obtain remote sensing imagery of Bordeaux-covered citrus canopies during the months of July, September, and November. We integrated spectral and texture information to construct a high-dimensional feature dataset and performed data downscaling and feature optimization. Furthermore, we established four machine learning models, namely, partial least squares regression (PLS), ridge regression (RR), ridge, random forest (RF), and support vector regression (SVR). Our objectives were to identify the most effective prediction model for estimating the SPAD (soil plant analysis development) value of Bordeaux-covered citrus canopies, assess the variation in prediction accuracy between fused features and individual features, and investigate the impact of Bordeaux solution on the spectral reflectance of the citrus canopy. The results showed that (1) the impact of Bordeaux mixture on citrus canopy reflectance bands ranked from the highest to the lowest as follows: near-infrared band at 840 nm, red-edge band at 730 nm, blue band at 450 nm, green band at 560 nm, and red band at 650 nm. (2) Fused feature models had better prediction ability than single-feature modeling, with an average R2 value of 0.641 for the four model test sets, improving by 0.117 and 0.039, respectively, compared with single-TF (texture feature) and -VI (vegetation index) modeling, and the test-set root-mean-square error (RMSE) was 2.594 on average, which was 0.533 and 0.264 lower than single-TF and -VI modeling, respectively. (3) Multiperiod data fusion effectively enhanced the correlation between features and SPAD values and consequently improved model prediction accuracy. Compared with accuracy based on individual months, R improved by 0.013 and 0.011, while RMSE decreased by 0.112 and 0.305. (4) The SVR model demonstrated the best performance in predicting citrus canopy SPAD under Bordeaux solution coverage, with R2 values of 0.629 and 0.658, and RMSE values of 2.722 and 2.752 for the training and test sets, respectively.

A novel approach using multispectral imaging for rapid development of seed pellet formulations to mitigate drought stress in alfalfa

Seed pelleting is an advanced technology that can improve seed sowing and germination under abiotic stresses such as drought, promoting sustainable agriculture. However, the development of conventional seed pelleting formulations is often characterized by prolonged timelines and significant labor costs. This study aimed to accelerate the development of water retention agents (WRAs) pelleting formulations for alfalfa seed under drought stress using multispectral imaging technology. The efficacy of these formulations was evaluated under drought stress conditions by measuring seedling physiological characteristics and collecting multispectral images. Results showed that WRA-pelleting formulations with HS0.5, BS1, and BS2 outperformed other WRA-pelleting formulations in terms of catalase, seedling length, superoxide dismutase, proline, and shoot length by principal component analysis. Additionally, backpropagation neural network, support vector machine, and random forest models were employed to identify various WRA-pelleting formulations using multispectral data from seedlings and cotyledons. All three models had an accuracy above 0.9 based on multispectral data from cotyledons, with support vector machine achieving the highest accuracy of 100%. The SHapley Additive exPlanations method was applied to explain the prediction mechanism of the support vector machine model and identified important features such as 970, 850, 880, 940, and 780 nm. Significant correlations were found between spectra and physiological indicators such as antioxidant, chlorophyll, and drought stress. In conclusion, our new approach has great potential to reduce the time and cost of developing seed pelleting formulations for mitigating drought stress.

Advances in multi-modal non-invasive imaging techniques in the diagnosis and treatment of polypoidal choroidal vasculopathy.

Polypoidal choroidal vasculopathy (PCV) is a disease characterized by subretinal pigment epithelium (RPE) orange-red polypoidal lesions and abnormal branching neovascular networks (BNNs). In recent years, various non-invasive imaging technologies have rapidly developed, especially the emergence of optical coherence tomography angiography (OCTA), multi-spectral imaging, and other technologies, which enable the observation of more features of PCV. In addition, these technologies are faster and less invasive compared to indocyanine green angiography (ICGA). Multi-modal imaging, which combined multiple imaging techniques, provides important references for the diagnosis and treatment of PCV with the assistance of regression models, deep learning, and other algorithms. In this study, we reviewed the non-invasive imaging techniques, multi-modal imaging diagnosis, and multi-scene therapeutic applications of PCV, with the aim of providing a reference for non-invasive multi-modal diagnosis and treatment of PCV.

Method for enhancing transmission image of breast obtained in visible and near-infrared bands

Multispectral transmission imaging is expected to be a new method for early screening of breast cancer. For the transmission images of the breast, which have a very low signal-to-noise ratio, the frame accumulation technique can be used to substantially improve the signal-to-noise ratio, but it will cause data redundancy. Terraced compression method not only reduces the redundancy of data, but also enhances the contrast of the image, so it is well suited for gray-scale compression of image. However, due to factors such as uneven tissue thickness and unevenly distributed illumination, the images contain trend item, which affect the effectiveness of the terraced compression method. Therefore, this paper obtains and eliminates the trend items contained in the images by polynomial fitting to ensure the effectiveness of the terraced compression method. The results of processing transmission images of breasts in the visible and near-infrared bands obtained from clinical trials show that the method of this paper can effectively remove trend item and enhance the contrast of the transmissive images. This can promote the application of multispectral imaging technology for early detection of breast cancer.

Multispectral Image Determination of Water Content in Aquilaria sinensis Based on Machine Learning

The real-time nondestructive monitoring of plant water content can enable operators to understand the water demands of crops in a timely manner and provide a reliable basis for precise irrigation. In this study, a method for rapid estimation of water content in Aquilaria sinensis using multispectral imaging was proposed. First, image registration and segmentation were performed using the Fourier–Mellin transform (FFT) and the fuzzy local information c-means clustering algorithm (FLICM). Second, the spectral features (SFs), texture features (TFs), and comprehensive features (CFs) of the image were extracted. Third, using the eigenvectors of the SFs, TFs, and CFs as input, a random forest regression model for estimating the water content of A. sinensis was constructed, respectively. Finally, the monarch butterfly optimization (MBO), Harris hawks optimization (HHO), and sparrow search algorithm (SSA) were used to optimize all models to determine the best estimation model. The results showed that: (1) 60%–80% soil water content is the most suitable for A. sinensis growth. Compared with waterlogging, drought inhibited A. sinensis growth more significantly. (2) FMT + FLICM could achieve rapid segmentation of discrete A. sinensis multispectral images on the basis of guaranteed accuracy. (3) The prediction effect of TFs was basically the same as that of SFs, and the prediction effect of CFs was higher than that of SFs and TFs, but this difference would decrease with the optimization of the RFR model. (4) Among all models, SSA-RFR_CFs had the highest accuracy, with an R2 of 0.8282. These results confirmed the feasibility and accuracy of applying multispectral imaging technology to estimate the water content of A. sinensis and provide a reference for the protection and cultivation of endangered precious tree species.

Monitoring the Severity of Rubber Tree Infected with Powdery Mildew Based on UAV Multispectral Remote Sensing

Rubber tree powdery mildew (PM) is one of the most devastating leaf diseases in rubber forest plantations. To prevent and control PM, timely and accurate detection is essential. In recent years, unmanned Aerial Vehicle (UAV) remote sensing technology has been widely used in the field of agriculture and forestry, but it has not been widely used to detect forest diseases. In this study, we propose a method to detect the severity of PM based on UAV low-altitude remote sensing and multispectral imaging technology. The method uses UAVs to collect multispectral images of rubber forest canopies that are naturally infected, and then extracts 19 spectral features (five spectral bands + 14 vegetation indices), eight texture features, and 10 color features. Meanwhile, Pearson correlation analysis and sequential backward selection (SBS) algorithm were used to eliminate redundant features and discover sensitive feature combinations. The feature combinations include spectral, texture, and color features and their combinations. The combinations of these features were used as inputs to the RF, BPNN, and SVM algorithms to construct PM severity models and identify different PM stages (Asymptomatic, Healthy, Early, Middle and Serious). The results showed that the SVM model with fused spectral, texture, and color features had the best performance (OA = 95.88%, Kappa = 0.94), as well as the highest recognition rate of 93.2% for PM in early stages.

The Development of Snapshot Multispectral Imaging Technology Based on Artificial Compound Eyes

In the present study, the advantages of multispectral imaging over hyperspectral imaging in real-time spectral imaging are briefly analyzed, and the advantages and disadvantages of snapshot spectral imaging and other spectral imaging technologies are briefly described. The technical characteristics of artificial compound eyes and multi-aperture imaging and the research significance of snapshot artificial compound eye multispectral imaging are also introduced. The classification and working principle of the snapshot artificial compound eye multispectral imaging system are briefly described. According to the realization method of the optical imaging system, the ACE snapshot multi-aperture multispectral imaging system is divided into plane and curved types. In the planar compound eye spectral imaging system, the technical progress of the multispectral imaging system based on the thin observation module by bound optics (TOMBO) architecture and the multispectral imaging system based on the linear variable spectral filter are introduced. At the same time, three curved multispectral imaging systems are introduced. Snapshot artificial compound eye multispectral imaging technology is also briefly analyzed and compared. The research results are helpful to comprehensively understand the research status of snapshot multispectral multi-aperture imaging technology based on artificial compound eyes and to lay the foundation for improving its comprehensive performance even further.

Monitoring of Wheat Fusarium Head Blight on Spectral and Textural Analysis of UAV Multispectral Imagery

Crop disease identification and monitoring is an important research topic in smart agriculture. In particular, it is a prerequisite for disease detection and the mapping of infected areas. Wheat fusarium head blight (FHB) is a serious threat to the quality and yield of wheat, so the rapid monitoring of wheat FHB is important. This study proposed a method based on unmanned aerial vehicle (UAV) low-altitude remote sensing and multispectral imaging technology combined with spectral and textural analysis to monitor FHB. First, the multispectral imagery of the wheat population was collected by UAV. Second, 10 vegetation indices (VIs)were extracted from multispectral imagery. In addition, three types of textural indices (TIs), including the normalized difference texture index (NDTI), difference texture index (DTI), and ratio texture index (RTI) were extracted for subsequent analysis and modeling. Finally, VIs, TIs, and VIs and TIs integrated as the input features, combined with k-nearest neighbor (KNN), the particle swarm optimization support vector machine (PSO-SVM), and XGBoost were used to construct wheat FHB monitoring models. The results showed that the XGBoost algorithm with the fusion of VIs and TIs as the input features has the highest performance with the accuracy and F1 score of the test set being 93.63% and 92.93%, respectively. This study provides a new approach and technology for the rapid and nondestructive monitoring of wheat FHB.

Multispectral drone imaging technology in agriculture

Unmanned aerial vehicles (UAVs) are used in various agricultural sectors to conduct important analyses and scans. They help identify bacteria, fungi, areas that require irrigation, and more. The agricultural industry in developed countries like Russia, the United States, and the European Union is widely considered to be the leading user of UAV technologies. Products with UAV engines are extensively integrated into these agricultural systems, essentially revolutionizing the agrarian sector. Drones play a crucial role in modern agricultural production through their three-dimensional and high-precision real-time analysis, measuring overall soil conditions, planting patterns, and ultimately providing vital data for optimal irrigation and nitrogen requirements.

Short-range multispectral imaging is an inexpensive, fast, and accurate approach to estimate biodiversity in a temperate calcareous grassland.

Image sensing technologies are rapidly increasing the cost-effectiveness of biodiversity monitoring efforts. Species differences in the reflectance of electromagnetic radiation can be used as a surrogate estimate plant biodiversity using multispectral image data. However, these efforts are often hampered by logistical difficulties in broad-scale implementation. Here, we investigate the utility of multispectral imaging technology from commercially available unmanned aerial vehicles (UAVs, or drones) in estimating biodiversity metrics at a fine spatial resolution (0.1-0.5cm pixel resolution) in a temperate calcareous grassland in Oxfordshire, UK. We calculate a suite of moments (coefficient of variation, standard deviation, skewness, and kurtosis) for the distribution of radiance from multispectral images at five wavelength bands (Blue 450 ± 16 nm; Green 560 ± 16 nm; Red 650 ± 16 nm; Red Edge 730 ± 16 nm; Near Infrared 840 ± 16 nm) and test their effectiveness at estimating ground-truthed biodiversity metrics from in situ botanical surveys for 37-1 × 1m quadrats. We find positive associations between the average coefficient of variation in spectral radiance and both the Shannon-Weiner and Simpson's biodiversity indices. Furthermore, the average coefficient of variation in spectral radiance is consistent and highly repeatable across sampling days and recording heights. Positive associations with biodiversity indices hold irrespective of the image recording height (2-8m), but we report reductions in estimates of spectral diversity with increases to UAV recording height. UAV imaging reduced sampling time by a factor of 16 relative to in situ botanical surveys. We demonstrate the utility of multispectral radiance moments as an indicator of biodiversity in this temperate calcareous grassland at a fine spatial resolution using a widely available UAV monitoring system with a coarse spectral resolution. The use of UAV technology with multispectral sensors has far-reaching potential to provide cost-effective and high-resolution monitoring of biodiversity.

Evaluation of insulator aging status based on multispectral imaging optimized by hyperspectral analysis

Composite insulators are widely used in transmission line and power equipment due to high mechanical strength and antifouling property. However, the aging problem of composite insulator caused by the degradations of organic materials is considered as the primary cause of common insulator faults. In this paper, an efficient aging evaluation method is proposed based on multispectral and hyperspectral imaging technologies. First, the hyperspectral analysis was performed to select the primary characteristic bands in 374–970 nm. Then, using a simplified multispectral imager, the aging degree of the samples were obtained by spatial clustering of gray values of multispectral images collected by the multispectral imager. In the first and non-optimized verification experiment, an aging status detection accuracy of 83 % was achieved. The proposed method is therefore a very promising attempt to apply spectral imaging technology to power equipment evaluation, which has great potential for on-site status monitoring of insulation materials.

Application of multispectral imaging combined with machine learning methods for rapid and non-destructive detection of zearalenone (ZEN) in maize

Maize is inevitably contaminated by zearalenone (ZEN) that will cause serious harm to human beings. In this study, multispectral imaging (MSI) technology combined with different machine learning methods were used to detect ZEN content in maize. The wavelengths that were most related to ZEN content in maize could be selected by genetic algorithm with back-propagation neural network (GA-BPNN). Our results showed that ZEN contamination level could be detected with the accuracy of 93.33 % by GA-BPNN method. In addition, for quantitative prediction of ZEN content GA-BPNN algorithm was the best method with the correlation coefficient (Rp), the root means square error (RMSEP), residual predictive deviation (RPD) and bias achieved to 0.95, 3.66 μg/kg, 5.39 and 1.55 μg/kg, respectively in prediction set. It can be concluded that multispectral imaging combined with machine learning was applicable for rapid measurement of ZEN content in maize.