Multispectral Scene Research Articles

SAM-HIT: A Simulated Annealing Multispectral to Hyperspectral Imagery Data Transformation

Space-borne hyperspectral imagery data are known for their high spectral resolution in a number of narrow wavelength intervals, which makes these data useful for mineral mapping. However, the available free-of-charge hyperspectral scenes cover only narrow and scattered geographic areas. In contrast, multispectral imagery scenes have a nearly complete spatial coverage and wider wavelength intervals. The low spectral resolution of the multispectral data, however, limits their efficiency in the mineral mapping of small geological massifs or hydrothermal alteration halos. The present contribution presents a new transformation tool (SAM-HIT) to simulate the hyperspectral sensor responses in unscanned areas based on partially overlapping hyperspectral and multispectral scenes. Simulation or prediction of the pseudohyperspectral data is here accomplished by using the simulated annealing linear optimization algorithm, which allows the lowest possible mismatch between the original and predicted data. The high visual and numerical correlation of the resultant data confirms the reliability of the newly adopted transformation. Further, the application of the SAM-HIT to a well-exposed part of the Egyptian basement complex with available hyperspectral data showed high concordance and nearly identical band signatures, opening a new outlook for mineral exploration in vast areas by a nearly automated cost-free means.

Ensemble Learning for Multispectral Scene Classification

In the recent decades, various techniques based on deep convolutional neural networks (DCNNs) have been applied to scene classification. Most of the techniques are established upon single-spectral images such that environmental conditions may greatly affect the quality of images in the visible (RGB) spectrum. One remedy for this downside is to merge the infrared (IR) with the visible spectrum for gaining the complementary information in comparison with the unimodal analysis. This paper incorporates the RGB, IR and near-infrared (NIR) images into a multispectral analysis for scene classification. For this purpose, two strategies are adopted. In the first strategy, each RGB, IR and NIR image is separately applied to DCNNs and then classified according to the output score of each network. In addition, an optimal decision threshold is obtained based on the same output score of each network. In the second strategy, three image components are extracted from each type of image using wavelet transform decomposition. Independent DCNNs are then trained on the image components of all the scene classes. Eventually, the final classification of the scene is accomplished through an appropriate ensemble architecture. The use of this architecture alongside a transfer learning approach and simple classifiers leads to lesser computational costs in small datasets. These experiments reveal the superiority of the proposed method over the state-of-the-art architectures in terms of the accuracy of scene classification.

Multispectral Scene Classification via Cross-Modal Knowledge Distillation

Scene classification is a fundamental task for numeral remote sensing applications, which aims to assign semantic labels to image patches. Although deep neural networks (DNN) demonstrated unique strength in scene classification, their performances are still limited due to the lack of training samples in the remote sensing field. Recent studies show that the performance of scene classification can be improved by taking advantage of the knowledge transferred from models pre-trained on RGB images. However, the modalities differences between input images hinder the knowledge transfer across models, especially when the input of the models has distinct spectral bands. To tackle the challenges, we propose a cross-modal knowledge distillation framework to improve the performance of multispectral scene classification by transferring the prior knowledge from teacher models pre-trained on RGB images to the student network with limited samples. Moreover, a teacher assistant (TA) network is introduced to further improve the classification performance by bridging the gap between the teacher and student networks. The proposed strategy is evaluated on models with multimodality inputs with distinct spectral bands and demonstrates superior performance as compared to the state-of-the-art methods.

Semantic segmentation of citrus-orchard using deep neural networks and multispectral UAV-based imagery

Accurately mapping farmlands is important for precision agriculture practices. Unmanned aerial vehicles (UAV) embedded with multispectral cameras are commonly used to map plants in agricultural landscapes. However, separating plantation fields from the remaining objects in a multispectral scene is a difficult task for traditional algorithms. In this connection, deep learning methods that perform semantic segmentation could help improve the overall outcome. In this study, state-of-the-art deep learning methods to semantic segment citrus-trees in multispectral images were evaluated. For this purpose, a multispectral camera that operates at the green (530–570 nm), red (640–680 nm), red-edge (730–740 nm) and also near-infrared (770–810 nm) spectral regions was used. The performance of the following five state-of-the-art pixelwise methods were evaluated: fully convolutional network (FCN), U-Net, SegNet, dynamic dilated convolution network (DDCN) and DeepLabV3 + . The results indicated that the evaluated methods performed similarly in the proposed task, returning F1-Scores between 94.00% (FCN and U-Net) and 94.42% (DDCN). It was also determined the inference time needed per area and, although the DDCN method was slower, based on a qualitative analysis, it performed better in highly shadow-affected areas. This study demonstrated that the semantic segmentation of citrus orchards is highly achievable with deep neural networks. The state-of-the-art deep learning methods investigated here proved to be equally suitable to solve this task, providing fast solutions with inference time varying from 0.98 to 4.36 min per hectare. This approach could be incorporated into similar research, and contribute to decision-making and accurate mapping of plantation fields.

NaSC-TG2: Natural Scene Classification With Tiangong-2 Remotely Sensed Imagery

Scene classification is one of the most important applications of remote sensing. Researchers have proposed various datasets and innovative methods for remote sensing scene classification in recent years. However, most of the existing remote sensing scene datasets are collected uniquely from a single data source: Google Earth. In addition, scenes in different datasets are mainly human-made landscapes with high similarity. The lack of richness and diversity of data sources limits the research and applications of remote sensing classification. This article describes a large-scale dataset named “NaSC-TG2,” which is a novel benchmark dataset for remote sensing natural scene classification built from Tiangong-2 remotely sensed imagery. The goal of this dataset is to expand and enrich the annotation data for advancing remote sensing classification algorithms, especially for the natural scene classification. The dataset contains 20 000 images, which are equally divided into ten scene classes. The dataset has three primary advantages: 1) it is large scale, especially in terms of the number of each class, and the numbers of scenes are evenly distributed; 2) it has a large number of intraclass differences and high interclass similarity, because all images are carefully selected from different regions and seasons; and 3) it offers natural scenes with novel spatial scale and imaging performance compared with other datasets. All images are acquired from the new generation of wideband imaging spectrometer of Tiangong-2. In addition to RGB images, the corresponding multispectral scene images are also provided. This dataset is useful in supporting the development and evaluation of classification algorithms, as demonstrated in the present study.

MSMatch: Semisupervised Multispectral Scene Classification With Few Labels

Supervised learning techniques are at the center of many tasks in remote sensing. Unfortunately, these methods, especially recent deep learning methods, often require large amounts of labeled data for training. Even though satellites acquire large amounts of data, labeling the data is often tedious, expensive and requires expert knowledge. Hence, improved methods that require fewer labeled samples are needed. We present MSMatch, the first semi-supervised learning approach competitive with supervised methods on scene classification on the EuroSAT benchmark dataset. We test both RGB and multispectral images and perform various ablation studies to identify the critical parts of the model. The trained neural network achieves state-of-the-art results on EuroSAT with an accuracy that is between 1.98% and 19.76% better than previous methods depending on the number of labeled training examples. With just five labeled examples per class we reach 94.53% and 95.86% accuracy on the EuroSAT RGB and multispectral datasets, respectively. With 50 labels per class we reach 97.62% and 98.23% accuracy. Our results show that MSMatch is capable of greatly reducing the requirements for labeled data. It translates well to multispectral data and should enable various applications that are currently infeasible due to a lack of labeled data. We provide the source code of MSMatch online to enable easy reproduction and quick adoption.

Optimized coded aperture for frugal hyperspectral image recovery using a dual-disperser system.

We present a novel acquisition scheme based on a dual-disperser architecture, which can reconstruct a hyperspectral datacube using many times fewer acquisitions than spectral bands. The reconstruction algorithm follows a quadratic regularization approach, based on the assumption that adjacent pixels in the scene share similar spectra, and, if they do not, this corresponds to an edge that is detectable on the panchromatic image. A digital micro-mirror device applies reconfigurable spectral-spatial filtering to the scene for each acquisition, and the filtering code is optimized considering the physical properties of the system. The algorithm is tested on simple multi-spectral scenes with 110 wavelength bands and is able to accurately reconstruct the hyperspectral datacube using only 10 acquisitions.

S-DolLion-MSVNN: A Hybrid Model for Developing the Super-Resolution Image From the Multispectral Satellite Image

Abstract Super-resolution offers a new image with high resolution from the low-resolution (LR) image that is highly employed for the numerous remote sensing applications. Most of the existing techniques for formation of the super-resolution image exhibit the loss of quality and deviation from the original multi-spectral LR image. Thus, this paper aims at proposing an efficient super-resolution method using the hybrid model. The hybrid model is developed using the support vector regression model and multi-support vector neural network (MSVNN), and the weights of the MSVNN is tuned optimally using the proposed algorithm. The proposed DolLion algorithm is the integration of the dolphin echolocation algorithm and lion optimization algorithm that exhibits better convergence and offers a global optimal solution. The experimentation is performed using the datasets taken from the multi-spectral scene images. The optimal and effective formation of the super-resolution image using the proposed hybrid model outperforms the existing methods, and the analysis using the second-derivative-like measure of enhancement (SDME) ensures that the proposed method is better and yields a maximum SDME of 67.6755 dB.

Multispectral Contrast of Archaeological Features: A Quantitative Evaluation

This study provides an evaluation of spectral responses of hollow ways in Upper Mesopotamia. Hollow ways were used for the transportation of animals, carts, and other moving agents for centuries. The aim is to show how the success of spectral indices varies in describing topologically simple features even in a seemingly homogeneous geographic unit. The variation is further highlighted under the changing precipitation regime. The methodology begins with an exploration of the relationship between the date of a multispectral scene and the visibility of hollow ways. The next step is to evaluate the impact of rainfall levels on numerous indices and to quantify spectral contrast. The contrast between a hollow way and its background is evaluated with Welch’s t-test and the association between precipitation regime and spectral responses of hollow ways are investigated with Correspondence Analysis and Fisher’s test. Results highlight an intrinsic relationship between the precipitation regime and the ways in which archaeological features reflects and/or emits electromagnetic energy. Next, the categorization of spectral indices based on different rainfall levels can be used as a guidance in future studies. Finally, the study suggests contrast becomes an even more fruitful concept as one moves from the spatial domain to the spectral domain.

RANUS: RGB and NIR Urban Scene Dataset for Deep Scene Parsing

In this letter, we present a data-driven method for scene parsing of road scenes to utilize single-channel near-infrared (NIR) images. To overcome the lack of data problem in non-RGB spectrum, we define a new color space and decompose the task of deep scene parsing into two subtasks with two separate CNN architectures for chromaticity channels and semantic masks. For chromaticity estimation, we build a spatially-aligned RGB-NIR image database (40k urban scenes) to infer color information from RGB-NIR spectrum learning process and leverage existing scene parsing networks trained over already available RGB masks. From our database, we sample key frames and manually annotate them (4k ground truth masks) to finetune the network into the proposed color space. Hence, the key contribution of this work is to replace multispectral scene parsing methods with a simple yet effective approach using single NIR images. The benefits of using our algorithm and dataset are confirmed in the qualitative and quantitative experiments.

First experience with Remote Sensing methods and selected sensors in the monitoring of mining areas – a case study of the Belchatow open cast mine

The paper presents satellite imagery from active SENTINEL-1A and passive SENTINEL-2A/2B sensors for their application in the monitoring of mining areas focused on detecting land changes. Multispectral scenes of SENTINEL-2A/2B have allowed for detecting changes in land-cover near the region of interest (ROI), i.e. the Szczercow dumping site in the Belchatow open cast lignite mine, central Poland, Europe. Scenes from SENTINEL-1A/1B satellite have also been used in the research. Processing of the SLC signal enabled creating a return intensity map in VV polarization. The obtained SAR scene was reclassified and shows a strong return signal from the dumping site and the open pit. This fact may be used in detection and monitoring of changes occurring within the analysed engineering objects.

Multi-Scale Segmentation Method Based on Binary Merge Tree and Class Label Information

This paper presents a newly developed segmentation algorithm for high-resolution remote sensing imagery. The new method is based on binary merge tree (BMT) and mainly comprises two parts, which are BMT construction and tree node selection. The first part is initialized by super-pixels, which are iteratively merged through using the well-known multi-resolution segmentation approach to complete BMT construction. The primary contribution of this paper resides in the second part, in which a new tree node selection criterion is developed. The new criterion is formulated by using spectral variance and class label cues in order to better describe intra-segment homogeneity. To validate the proposed strategy, four multi-spectral scenes captured by two Chinese satellites, ZiYuan-3 and GaoFen-2, were adopted for segmentation experiments. The results indicated that our method out-performed some state-of-the-art algorithms in terms of segmentation accuracy.

Colored Coded Aperture Design in Compressive Spectral Imaging via Minimum Coherence

Colored coded aperture optimization in compressive spectral imaging is discussed. Based on the analysis of the coherence of the underlying sensing matrix, a general family of codes is derived. These designs lead to reconstructions of multispectral scenes of better quality than the ones obtained using the traditional random black and white coded apertures. The approach used in this work exploits the structure of the sensing matrices and reduces the problem of the design of a large-scale matrix to a subset of substantially smaller problems for which it is possible to obtain a closed form solution, leading to fast design algorithms.

Validation of a Simplified Model to Generate Multispectral Synthetic Images

A new procedure to assess the quality of topographic correction (TOC) algorithms applied to remote sensing imagery was previously proposed by the authors. This procedure was based on a model that simulated synthetic scenes, representing the radiance an optical sensor would receive from an area under some specific conditions. TOC algorithms were then applied to synthetic scenes and the resulting corrected scenes were compared with a horizontal synthetic scene free of topographic effect. This comparison enabled an objective and quantitative evaluation of TOC algorithms. This approach showed promising results but had some shortcomings that are addressed herein. First, the model, originally built to simulate only broadband panchromatic scenes, is extended to multispectral scenes in the visible, near infrared (NIR), and short wave infrared (SWIR) bands. Next, the model is validated by comparing synthetic scenes with four Satellite pour l'Observation de la Terre 5 (SPOT5) real scenes acquired on different dates and different test areas along the Pyrenees mountain range (Spain). The results obtained show a successful simulation of all the spectral bands. Therefore, the model is deemed accurate enough for its purpose of evaluating TOC algorithms.

Lattice algebra approach to multispectral analysis of ancient documents

This paper introduces a lattice algebra procedure that can be used for the multispectral analysis of historical documents and artworks. Assuming the presence of linearly mixed spectral pixels captured in a multispectral scene, the proposed method computes the scaled min- and max-lattice associative memories to determine the purest pixels that best represent the spectra of single pigments. The estimation of fractional proportions of pure spectra at each image pixel is used to build pigment abundance maps that can be used for subsequent restoration of damaged parts. Application examples include multispectral images acquired from the Archimedes Palimpsest and a Mexican pre-Hispanic codex.

Direct measurement of ocean waves velocity field from a single SPOT-5 dataset

We present a method based on space-borne optical imagery from the SPOT5 satellite to directly measure the phase velocity fields of ocean waves. The panchromatic and multispectral scenes acquired by SPOT5 the same day on the same area are not strictly superimposable due to the different locations of the CCDs (Charged Coupled Device) in the focal plane of the instrument. In this manuscript, we propose a method that exploits the temporal lag that exists between the panchromatic and multispectral scenes to measure the ocean wave velocity fields. We firstly discuss the principle and the methodology. Then, we apply it offshore La Reunion Island. Finally, we compare and discuss the results against a swell propagation model. Our method is proven reliable and can be immediately extended to other push-broom sensors.

Crisp and fuzzy competitive learning networks for supervised classification of multispectral IRS scenes

Crisp and fuzzy competitive learning network schemes have been designed for classification of multispectral IRS-1B satellite images. For supervised learning, an extension of competitive learning network with a Grossberg layer, sometimes known as a 'forward only' Counter-propagation Network (CPN) has been used. The 'concept of winner' of a classical Kohonen's network has been fuzzified in this model. This model is found to yield much better accuracy than the crisp Kohonen's network and marginally better accuracy than the Maximum Likelihood Classifier. The results are discussed.

Multivariate Gaussian MRF for multispectral scene segmentation and anomaly detection

Gaussian Markov random field texture models and multivariate parametric clustering algorithms have been applied extensively for segmentation, restoration, and anomaly detection of single-band and multispectral imagery, respectively. The present work extends and combines these previous efforts to demonstrate joint spatial-spectral modeling of multispectral imagery, a multivariate (vector observations) GMRF texture model is employed. Algorithms for parameter estimation and image segmentation are discussed, and a new anomaly detection technique is developed. The model is applied to imagery from the Daedalus sensor. Image segmentation results from test images are discussed and compared to spectral clustering results. The test images are collages, with known texture boundaries constructed from larger data cubes. Anomaly detection results for two Daedalus images are also presented, assessed using receiver operating characteristic (ROC) performance curves, and compared to spectral clustering models. It is demonstrated that even the simplest first-order isotropic texture models provide significant improvement in image segmentation and anomaly detection over pure spectral clustering for the data sets examined. The sensitivity of anomaly detection performance to the choice of parameter estimation method and to the number of texture segments is examined for one example data set.

Multiclass spectral decomposition of remotely sensed scenes by selective pixel unmixing

Linear pixel unmixing is a straightforward and efficient approach to the spectral decomposition of multichannel remotely sensed scenes. A main drawback to its utilization in operational cases, however, is that the number of spectral components that can be correctly treated must be less or equal to the scene dimensionality (the so-called condition of identifiability). To overcome the limitations deriving from this condition, a two-step strategy is currently proposed for application to each scene pixel. Provided that many spectral end-members are available, a subset with a prefixed number of end-members that optimally decompose the candidate pixel is first selected by a procedure based on the Gramm-Schmidt orthogonalization process. This restricted subset is then employed for conventional linear pixel unmixing. The final result is the decomposition of the multispectral scene into all the end-members considered while reducing the residual errors deriving from interclass spectral variability. The new procedure has been tested in three case studies representative of different environmental situations and data sets. The results of these experiments, compared to those of a conventional procedure, show that the new method identifies more clearly the spectral signal associated to all scene components and significantly reduces (20-30%) the residual error of the decomposition process. This is confirmed by further tests using synthetic scenes that are linear combinations of known end-members. In these cases, the reduction of the residual error by the new method is much higher (up to 70-80%) and the abundance images produced are more accurate estimates of the real components.

Component analysis of spatial and spectral patterns in multispectral images II Entropy minimization

In Part I [J. Opt. Soc. Am. A 4, 2101 (1987)] of this series, we developed a method for estimating both spatial patterns and spectral curves of components in a multispectral scene. This method does not need spatial and spectral information about the components but only multispread imagery data. The estimation is given as a feasible solution set satisfying the nonnegativity constraint for density and spectral response for all components at all pixels. In this paper, we estimate unique solutions for both the component patterns and the spectra from the feasible solution set. The solution is given by optimizing an entropy minimization criterion. This criterion enhances the spectral or spatial features of individual components. Two experimental results are shown to demonstrate the effectiveness of this method with biological and cytochemical specimens. The limitations of this method for unique pattern estimation are also discussed.