Multispectral Information Research Articles

Remote Sensing Image Generation Based on Attention Mechanism and VAE-MSGAN for ROI Extraction

A variety of deep learning approaches have been applied to region of interest (ROI) extraction, which is a fundamental task in the field of remote sensing image (RSI) processing. However, the unbalanced distribution of positive and negative samples in most RSIs greatly restricts the performance of these deep learning-based methods. In this study, a data augmentation method based on variational autoencoder-multiscale generative adversarial network (VAE-MSGAN) with spatial and channelwise attention (SCA) is proposed to balance the sample distribution and improve the subsequent ROI extraction results. First, we combine the original multispectral information with handcrafted texture features to make full use of the low-level visual features of RSIs. We then design a VAE-MSGAN to generate realistic RSIs with high quality and diversity. Specifically, in the generator construct, SCA blocks are introduced to adaptively recalibrate the varying importance of different channels and spatial regions. We also build a multiscale discriminator architecture to improve the visual quality of the generated samples. Finally, we compare the ROI extraction results before and after the augmentation. Our experimental results demonstrate that the proposed method can not only improve the performance of ROI extraction but also be superior to other classical generative methods.

Evaluating single and multi-date Landsat classifications of land-cover in a seasonally dry tropical forest

Accurate information on the land cover is crucial for efficient monitoring and development of environmental studies in the Brazilian Caatinga forest. It is one of the largest and most biodiverse dry forests on the planet. Distinguishing different patterns of land cover through medium spatial-resolution remote sensing, such as the Landsat image series, is challenging to Caatinga due to heterogeneous land cover, complex climate-soil - vegetation interactions, and anthropogenic disturbance. Two remote sensing approaches have a high potential for accurate and efficient land-cover mapping in Caatinga: single and multi-date imagery. The heterogeneity of the land cover of this environment can contribute to a better performance of multispectral approaches, although it is usually applied for single-date images. In a land-cover mapping effort in Caatinga, the temporal factor gains relevance, and the use of time series can bring advantages, but, in general, this approach uses vegetation index, losing multispectral information. This manuscript assesses the accuracies and advantages of single-date multispectral and multi-date Normalized Difference Vegetation Index (NDVI) approaches in land-cover classification. Both approaches use the Random Forest method, and the results are evaluated based on samples collected during field surveys. Results indicate that land-cover classification obtained from multi-date NDVI performs better (overall accuracy of 88.8% and kappa of 0.86) than single-date multispectral data (overall accuracy of 81.4% and kappa coefficient of 0.78). The Z-test indicated that the difference in performance between the two approaches was statistically significant. The lower performance observed for single-date multispectral classification is due to similarities in spectral responses for targets of deciduous vegetation that lose their foliage and can be misread as non-vegetated areas. Meanwhile, an accurate classification by time series of plant clusters in seasonal forests allows incorporating seasonal variability of land-cover classes during the rainy and dry seasons, as well as transitions between seasons. The most important variables that contributed to the accuracy were the red, Near Infrared (NIR) and Short-Wave Infrared (SWIR) bands in single-date multispectral classification and the months in the dry season were the most relevant in multi-date NDVI classification.

Non-invasive diagnostic investigation at the Bishop’s Palace of Frascati: an integrated approach

<p>Artistic surfaces at the Bishop’s Palace of Frascati have been investigated by an integrated approach involving different non-invasive diagnostic techniques. A LIF (Laser Induced Fluorescence) scanning system worked in synergy with the RGB-ITR ((Red Green and Blue – Imaging Topological Radar) 3D laser scanner and the<em> SfM</em> (Structure from Motion) technique for the 3D photogrammetric reconstruction. The presented case study shows how 3D multispectral information can reveal and locate previous restoration actions and deterioration processes as support for conservation, research and dissemination purposes.</p>

Development of Nationwide Road Quality Map: Remote Sensing Meets Field Sensing.

In this study, we measured the in situ international roughness index (IRI) for first-degree roads spanning more than 1300 km in East Azerbaijan Province, Iran, using a quarter car (QC). Since road quality mapping with in situ measurements is a costly and time-consuming task, we also developed new equations for constructing a road quality proxy map (RQPM) using discriminant analysis and multispectral information from high-resolution Sentinel-2 images, which we calibrated using the in situ data on the basis of geographic information system (GIS) data. The developed equations using optimum index factor (OIF) and norm R provide a valuable tool for creating proxy maps and mitigating hazards at the network scale, not only for primary roads but also for secondary roads, and for reducing the costs of road quality monitoring. The overall accuracy and kappa coefficient of the norm R equation for road classification in East Azerbaijan province are 65.0% and 0.59, respectively.

Data-Fusion-Based Two-Stage Cascade Framework for Multimodality Face Anti-Spoofing

Existing face anti-spoofing models using deep learning for multimodality data suffer from low generalization in the case of using variety of presentation attacks, such as 2-D printing and high-precision 3-D face masks. One of the main reasons is that the nonlinearity of multispectral information used to preserve the intrinsic attributes between a real and a fake face is not well extracted. To address this issue, we propose a multimodility data-based two-stage cascade framework for face anti-spoofing. The proposed framework has two advantages. First, we design a two-stage cascade architecture that can selectively fuse low-level and high-level features from different modalities to improve feature representation. Second, we use multimodality data to construct a distance-free spectral on RGB and infrared to augment the nonlinearity of data. The presented data fusion strategy is different from popular fusion approaches, since it can strengthen discrimination ability of network models on physical attribute features than identity structure features under certain constraints. In addition, a multiscale patch-based weighted fine-tuning strategy is designed to learn each specific local face region. The experimental results show that the proposed framework achieves better performance than other state-of-the-art methods on both benchmark data sets and self-established data sets, especially on multimaterial masks spoofing.

Multi-sensor imagery rectification and registration for herbicide testing

The use of multi-spectral sensors has been focused on several agricultural tasks, yet it is necessary to further assess this approach to achieve sufficient precision to carry out adequately these. Metric information from these images is traditionally derived by photogrammetric techniques, but with a major limitation: photographed objects should be static while the photographs are being taken, but plants are generally in movement because of wind and this causes the photogrammetric process to be unable to generate the necessary information to make any metric measurement. To bypass this, metric information can be derived via rectification, using only one photograph.This work aims to develop a band co-registration method with agricultural purposes, based on rectified images taken from different sensors usually mounted on UAVs or terrestrial vehicles, studying its accuracy in a quantitative way. All multispectral information co-registered in a precise way will allow the calculation or development of new radiometric and even geometric indices that will help to improve efficiency in many tasks related to agriculture.Images taken from a multi-spectral (green, near infra-red, red and red edge) and a thermal camera were used to apply the developed methodology. First, a digital elevation model describing the displacement produced by distortion due to the sensor lens was obtained and applied to each of the studied pictures to correct this distortion. Then, distortion due to conic perspective present in the photographs was corrected, taking into account the homology relationship between the photographed object and the picture. To carry out these tasks, several computers programs were developed. Subsequently, the edges of the five bands corresponding to 250 plants were digitalised and their areas were measured. Furthermore, the intersection of the five bands of each plant was calculated, and an index (AI) indicating the fraction of the area of each band, which was out of the common area edge of the five bands, was calculated for each plant. The average value of this index for each band ranged from 0.22 to 0.24, with no statistically significant differences between them, indicating a high accuracy of the proposed methodology.

A Temporal-Spectral-Based Squeeze-and- Excitation Feature Fusion Network for Motor Imagery EEG Decoding.

Motor imagery (MI) electroencephalography (EEG) decoding plays an important role in brain-computer interface (BCI), which enables motor-disabled patients to communicate with the outside world via external devices. Recent deep learning methods, which fail to fully explore both deep-temporal characterizations in EEGs itself and multi-spectral information in different rhythms, generally ignore the temporal or spectral dependencies in MI-EEG. Also, the lack of effective feature fusion probably leads to redundant or irrelative information and thus fails to achieve the most discriminative features, resulting in the limited MI-EEG decoding performance. To address these issues, in this paper, a MI-EEG decoding framework is proposed, which uses a novel temporal-spectral-based squeeze-and-excitation feature fusion network (TS-SEFFNet). First, the deep-temporal convolution block (DT-Conv block) implements convolutions in a cascade architecture, which extracts high-dimension temporal representations from raw EEG signals. Second, the multi-spectral convolution block (MS-Conv block) is then conducted in parallel using multi-level wavelet convolutions to capture discriminative spectral features from corresponding clinical subbands. Finally, the proposed squeeze-and-excitation feature fusion block (SE-Feature-Fusion block) maps the deep-temporal and multi-spectral features into comprehensive fused feature maps, which highlights channel-wise feature responses by constructing interdependencies among different domain features. Competitive experimental results on two public datasets demonstrate that our method is able to achieve promising decoding performance compared with the state-of-the-art methods.

Land cover-adjusted index for the former Aral Sea using Landsat images

The Aral Sea was the fourth largest inland lake on the globe until 1960, with a surface area of about 68,000 km2. Mainly, the huge irrigation projects in many parts of its transboundary catchment were responsible for the catastrophic desiccation and ecological crises of the Aral Sea after second part of 20th century. Ecological crisis surrounding the Aral Sea (lake) regions is one of the critical environmental problems of Central Asia. As a result, monitoring of desertification processes and determining the aerosol concentration in the atmosphere are highly relevant for any attempts to mitigate environmental changes in the Aral Sea basin. Remote sensing is the most appropriate method for studying desertification and dust storms as it easily covers large areas with a high spatial and temporal resolution. Satellite images provide detailed multispectral information about the earth’s surface features, which proves invaluable for the characterization of vegetation, soil, water, and landforms at different scales. Vegetation cover, biomass, and soil properties were analyzed with remote sensing methods (NDVI, SDVI). It is emphasized that vegetation indices have little sensitivity at low leaf area which is common to all desert ecosystems.

Mosaic Convolution-Attention Network for Demosaicing Multispectral Filter Array Images

This paper presents a mosaic convolution-attention network (MCAN) for demosaicing spectral mosaic images captured using multispectral filter array (MSFA) imaging sensors. MSFA-based multispectral imaging systems acquire multispectral information of a scene in a single snap-shot operation. A complete multispectral image is reconstructed by demosaicing an MSFA-based spectral mosaic image. To avoid aliasing and artifacts in demosaicing, we utilize joint spatial-spectral correlation in a raw mosaic image. The proposed MCAN includes a mosaic convolution module (MCM) and a mosaic attention module (MAM). The MCM extracts features via a learning approach with a margin between splitting the periodic spectral mosaic and keeping the underlying spatial information of the raw image. Based on the strategy of position-sensitive weight sharing, MCM assigns the same weight to pixels with the same relative position in an MSFA. The MAM uses a position-sensitive feature aggregation strategy to describe the loading of mosaic patterns within the feature maps, which gradually reduces mosaic distortion through the attention mechanism. The experimental results on synthetic as well as real-world data show that the proposed scheme outperforms state-of-the-art methods in terms of spatial details and spectral fidelity.

Big Data and Machine Learning With Hyperspectral Information in Agriculture

Hyperspectral and multispectral information processing systems and technologies have demonstrated its usefulness for the improvement of agricultural productivity and practices by providing useful information to farmers and crop managers on the factors affecting crop status and growth. These technologies are widely used in a range of agriculture applications such as crop management, crop yield forecasting, crop disease detection, and the monitoring of agriculture land usage, water, and soil conditions. Hyperspectral information sensing can acquire several hundred spectral bands that cover the electromagnetic spectrum of an observational scene in a single acquisition. The resulting hyperspectral data cube contains a large volume of spatial and spectral information. The hyperspectral sequence of images or video further increases the data generation velocity and volume which lead to the Big data challenges particularly in agricultural remote sensing applications. This paper is structured to first give a comprehensive review of representative studies to provide insights into significant research efforts in agriculture using Big data, machine learning and deep learning with the focus on frameworks or architectures, information processing and analytics with hyperspectral and multispectral data. The potential for utilizing Big data, machine learning and deep learning for hyperspectral and multispectral data in agriculture is very promising. The paper then further explores the potential of using ensemble machine learning and scalable parallel discriminant analysis which takes into consideration the spatial and spectral components for Big data in agriculture. To the best of our knowledge, no similar review study on agriculture with Big data, machine learning and deep learning for hyperspectral and multispectral information processing has been reported. Furthermore, the potential of ensemble machine learning and scalable parallel discriminant analysis has not been explored in agriculture information processing. Experiments and data analytics have been performed on hyperspectral data from agriculture for validation. The results have shown the good performance of our approach.

Detecting of Overshooting Cloud Tops via Himawari-8 Imagery Using Dual Channel Multiscale Deep Network

The occurrence of overshooting cloud tops can cause extremely severe weather such as strong winds and heavy rainfalls. The traditional overshooting cloud top (OT) detection methods not only need to choose a reasonable threshold, it is also very hard to make full advantage of the multispectral information of cloud images. These make small-scale OT detection very difficult with poor accuracy of OT boundary determination. In order to utilize the multispectral information of Himawari-8 satellite cloud images, in this article, we propose a method for detecting OT based on the dual channel multiscale deep network (DCMSDN). The brightness temperature of infrared window and the difference of brightness temperature between the infrared window and water vapor window are used as dual channel inputs, respectively. Then, DCMSDN introduces a multiscale prediction module to improve the accuracy of small target detection, which makes the network more suitable for the detection of the OT with small spatial scale. Experimental results indicate that the proposed method provides competitive performance with acceptable computational efficiency. Specifically, for the quantitative indicators of OTs detection, our approach achieves the accuracy of 89.36%, the precision of 95.63%, the recall of 88.90%, and the F1-measure of 91.61% for the test cloud images, which outperforms that of comparative methods.

Desertification Detection Using an Improved Variational Autoencoder-Based Approach Through ETM-Landsat Satellite Data

The accurate land cover change detection is critical to improve the landscape dynamics analysis and mitigate desertification problems efficiently. Desertification detection is a challenging problem because of the high degree of similarity between some desertification cases and like-desertification phenomena, such as deforestation. This article provides an effective approach to detect deserted regions based on Landsat imagery and variational autoencoder (VAE). The VAE model, as a deep learning-based model, has gained special attention in features extraction and modeling due to its distribution-free assumptions and superior nonlinear approximation. Here, a VAE approach is applied to spectral signatures for detecting pixels affected by the land cover change. The considered features are extracted from multitemporal images and include multispectral information, and no prior image segmentation is required. The proposed method was evaluated on the publicly available remote sensing data using multitemporal Landsat optical images taken from the freely available Landsat program. The arid region around Biskra in Algeria is selected as a study area since it is well-known that desertification phenomena strongly influence this region. The VAE model was evaluated and compared with restricted Boltzmann machines, deep learning model, and binary clustering algorithms, including Agglomerative, BIRCH, expected maximization, k-mean clustering algorithms, and one-class support vector machine. The comparative results showed that the VAE consistently outperformed the other models for detecting changes to the land cover, mainly deserted regions. This study also showed that VAE outperformed the state-of-the-art algorithms.

Three multispectral configurations of a snapshot kaleidoscope-based camera in long wavelength infrared spectral band

In the field of spectral imaging, numerous instruments use scanning-based technologies. However, the temporal dimension of these systems, whether to scan the spectrum or scan the scene, can be an issue for some applications. This is particularly the case when trying to observe and identify rapid temporal variations in a fixed scene or detecting objects of interest when moving. In this case, it is suitable to observe the desired spectral information of the scene simultaneously, and so-called snapshot systems have been thus investigated. In this paper, we study the ability of a kaleidoscope-based multiview camera to acquire multispectral information in the long wavelength infrared. Several strategies and technologies will be compared to add the spectral function inside the different blocks of a kaleidoscope-based camera: the front lens, the kaleidoscope, or the reimaging lens. The studied camera uses an uncooled infrared detector and thus must deal with the issue of having a large aperture.

LEARNING SUPER-RESOLUTION FOR SENTINEL-2 IMAGES WITH REAL GROUND TRUTH DATA FROM A REFERENCE SATELLITE

Abstract. Copernicus program via its Sentinel missions is making earth observation more accessible and affordable for everybody. Sentinel-2 images provide multi-spectral information every 5 days for each location. However, the maximum spatial resolution of its bands is 10m for RGB and near-infrared bands. Increasing the spatial resolution of Sentinel-2 images without additional costs, would make any posterior analysis more accurate. Most approaches on super-resolution for Sentinel-2 have focused on obtaining 10m resolution images for those at lower resolutions (20m and 60m), taking advantage of the information provided by bands of finer resolutions (10m). Otherwise, our focus is on increasing the resolution of the 10m bands, that is, super-resolving 10m bands to 2.5m resolution, where no additional information is available. This problem is known as single-image super-resolution and deep learning-based approaches have become the state-of-the-art for this problem on standard images. Obviously, models learned for standard images do not translate well to satellite images. Hence, the problem is how to train a deep learning model for super-resolving Sentinel-2 images when no ground truth exist (Sentinel-2 images at 2.5m). We propose a methodology for learning Convolutional Neural Networks for Sentinel-2 image super-resolution making use of images from other sensors having a high similarity with Sentinel-2 in terms of spectral bands, but greater spatial resolution. Our proposal is tested with a state-of-the-art neural network showing that it can be useful for learning to increase the spatial resolution of RGB and near-infrared bands of Sentinel-2.

Towards Circumpolar Mapping of Arctic Settlements and Infrastructure Based on Sentinel-1 and Sentinel-2

Infrastructure expands rapidly in the Arctic due to industrial development. At the same time, climate change impacts are pronounced in the Arctic. Ground temperatures are, for example, increasing as well as coastal erosion. A consistent account of the current human footprint is needed in order to evaluate the impact on the environments as well as risk for infrastructure. Identification of roads and settlements with satellite data is challenging due to the size of single features and low density of clusters. Spatial resolution and spectral characteristics of satellite data are the main issues regarding their separation. The Copernicus Sentinel-1 and -2 missions recently provided good spatial coverage and at the same time comparably high pixel spacing starting with 10 m for modes available across the entire Arctic. The purpose of this study was to assess the capabilities of both, Sentinel-1 C-band Synthetic Aperture Radar (SAR) and the Sentinel-2 multispectral information for Arctic focused mapping. Settings differ across the Arctic (historic settlements versus industrial, locations on bedrock versus tundra landscapes) and reference data are scarce and inconsistent. The type of features and data scarcity demand specific classification approaches. The machine learning approaches Gradient Boosting Machines (GBM) and deep learning (DL)-based semantic segmentation have been tested. Records for the Alaskan North Slope, Western Greenland, and Svalbard in addition to high-resolution satellite data have been used for validation and calibration. Deep learning is superior to GBM with respect to users accuracy. GBM therefore requires comprehensive postprocessing. SAR provides added value in case of GBM. VV is of benefit for road identification and HH for detection of buildings. Unfortunately, the Sentinel-1 acquisition strategy is varying across the Arctic. The majority is covered in VV+VH only. DL is of benefit for road and building detection but misses large proportions of other human-impacted areas, such as gravel pads which are typical for gas and oil fields. A combination of results from both GBM (Sentinel-1 and -2 combined) and DL (Sentinel-2; Sentinel-1 optional) is therefore suggested for circumpolar mapping.

Narrow-to-Broadband Conversion for Albedo Estimation on Urban Surfaces by UAV-Based Multispectral Camera

Urban surface albedo is important for investigating urban surface–atmosphere radiative heat exchanges. For modeling surface energy balance (SEB) at local and neighborhood scales, ground or unmanned aerial vehicle (UAV)-based multispectral remote sensing (RS) can be used to obtain high-spatial-resolution multispectral information for both horizontal and vertical urban surfaces. The existing narrow-to-broadband (NTB) conversion models, developed for satellite/high-altitude observation and large homogeneous rural/vegetated/snow zones, may not be suitable for downscaling to the local and neighborhood scales or the urban complex texture. We developed three NTB models following published methodologies for three common UAV-based multispectral cameras according to Sample_D, a sample group of extensive spectral albedos of artificial urban surfaces, and evaluated their performance and sensitivities to solar conditions and surface material class. The proposed models were validated with independent samples (Sample_V). A model considering albedo physics was improved by multiplying different variables with respect to the camera (termed as “Model_phy_reg”), which initially proved to be the most accurate with a root mean square error of up to 0.02 for Sample_D and approximately 0.029 for Sample_V, meeting the required accuracy of total shortwave albedo for SEB modeling. The accuracy of Model_phy_reg was not much prone to the solar conditions.

The potential of UAV-borne spectral and textural information for predicting aboveground biomass and N fixation in legume-grass mixtures.

Organic farmers, who rely on legumes as an external nitrogen (N) source, need a fast and easy on-the-go measurement technique to determine harvestable biomass and the amount of fixed N (NFix) for numerous farm management decisions. Especially clover- and lucerne-grass mixtures play an important role in the organic crop rotation under temperate European climate conditions. Multispectral sensors mounted on unmanned aerial vehicles (UAVs) are new promising tools for a non-destructive assessment of crop and grassland traits on large and remote areas. One disadvantage of multispectral information and derived vegetations indices is, that both ignore spatial relationships of pixels to each other in the image. This gap can be filled by texture features from a grey level co-occurrence matrix. The aim of this multi-temporal field study was to provide aboveground biomass and NFix estimation models for two legume-grass mixtures through a whole vegetation period based on UAV multispectral information. The prediction models covered different proportions of legumes (0-100% legumes) to represent the variable conditions in practical farming. Furthermore, the study compared prediction models with and without the inclusion of texture features. As multispectral data usually suffers from multicollinearity, two machine learning algorithms, Partial Least Square and Random Forest (RF) regression, were used. The results showed, that biomass prediction accuracy for the whole dataset as well as for crop-specific models were substantially improved by the inclusion of texture features. The best model was generated for the whole dataset by RF with an rRMSE of 10%. For NFix prediction accuracy of the best model was based on RF including texture (rRMSEP = 18%), which was not consistent with crop specific models.

A multispectral feature fusion network for robust pedestrian detection

The multispectral information, including both visible information and infrared information, can describe the detection target in a comprehensive manner. The deep learning (DL)-based detectors that fuse the multispectral features can detect pedestrians robustly in various environments. Therefore, this paper puts forward a robust multispectral feature fusion network (MSFFN) for pedestrian detection, which fully integrates the features extracted from visible light and infrared channels. Specifically, multiscale semantic features were extracted by two core modules, namely, multiscale feature extraction of visible images (MFEV) and multiscale feature extraction of infrared images (MFEI), and fused by the improved YOLOv3 network for pedestrian recognition. Through experiments on the KAIST dataset, it is proved that the MSFFN model can detect pedestrians more accurately than both MFEV and MFEI over daytime and nighttime images on multiple scales. The experimental results on the KAIST multispectral data set in the last section showed that our proposed MFMFN model was superior to a number of state-of-the-art multispectral pedestrian detectors methods in accuracy and speed. The model was also found to strike a good balance between accuracy and speed, and perform excellently on small input images. The research results shed important new light on the design of self-driving vehicles.

Comparison of Machine Learning Algorithms for Wildland-Urban Interface Fuelbreak Planning Integrating ALS and UAV-Borne LiDAR Data and Multispectral Images

Controlling vegetation fuels around human settlements is a crucial strategy for reducing fire severity in forests, buildings and infrastructure, as well as protecting human lives. Each country has its own regulations in this respect, but they all have in common that by reducing fuel load, we in turn reduce the intensity and severity of the fire. The use of Unmanned Aerial Vehicles (UAV)-acquired data combined with other passive and active remote sensing data has the greatest performance to planning Wildland-Urban Interface (WUI) fuelbreak through machine learning algorithms. Nine remote sensing data sources (active and passive) and four supervised classification algorithms (Random Forest, Linear and Radial Support Vector Machine and Artificial Neural Networks) were tested to classify five fuel-area types. We used very high-density Light Detection and Ranging (LiDAR) data acquired by UAV (154 returns·m−2 and ortho-mosaic of 5-cm pixel), multispectral data from the satellites Pleiades-1B and Sentinel-2, and low-density LiDAR data acquired by Airborne Laser Scanning (ALS) (0.5 returns·m−2, ortho-mosaic of 25 cm pixels). Through the Variable Selection Using Random Forest (VSURF) procedure, a pre-selection of final variables was carried out to train the model. The four algorithms were compared, and it was concluded that the differences among them in overall accuracy (OA) on training datasets were negligible. Although the highest accuracy in the training step was obtained in SVML (OA=94.46%) and in testing in ANN (OA=91.91%), Random Forest was considered to be the most reliable algorithm, since it produced more consistent predictions due to the smaller differences between training and testing performance. Using a combination of Sentinel-2 and the two LiDAR data (UAV and ALS), Random Forest obtained an OA of 90.66% in training and of 91.80% in testing datasets. The differences in accuracy between the data sources used are much greater than between algorithms. LiDAR growth metrics calculated using point clouds in different dates and multispectral information from different seasons of the year are the most important variables in the classification. Our results support the essential role of UAVs in fuelbreak planning and management and thus, in the prevention of forest fires.

Ultralow-light-level color image reconstruction using high-efficiency plasmonic metasurface mosaic filters

As single-photon imaging becomes progressively more commonplace in sensing applications such as low-light-level imaging, three-dimensional profiling, and fluorescence imaging, there exist a number of fields where multispectral information can also be exploited, e.g., in environmental monitoring and target identification. We have fabricated a high-transmittance mosaic filter array, where each optical filter was composed of a plasmonic metasurface fabricated in a single lithographic step. This plasmonic metasurface design utilized an array of elliptical and circular nanoholes, which produced enhanced optical coupling between multiple plasmonic interactions. The resulting metasurfaces produced narrow bandpass filters for blue, green, and red light with peak transmission efficiencies of 79%, 75%, and 68%, respectively. After the three metasurface filter designs were arranged in a 64 × 64 format random mosaic pattern, this mosaic filter was directly integrated onto a CMOS single-photon avalanche diode detector array. Color images were then reconstructed at light levels as low as approximately 5 photons per pixel, on average, via the simultaneous acquisition of low-photon multispectral data using both three-color active laser illumination and a broadband white-light illumination source.