Dimensionality Reduction Of Hyperspectral Data Research Articles

A Study on Dimensionality Reduction and Parameters for Hyperspectral Imagery Based on Manifold Learning.

With the rapid advancement of remote-sensing technology, the spectral information obtained from hyperspectral remote-sensing imagery has become increasingly rich, facilitating detailed spectral analysis of Earth's surface objects. However, the abundance of spectral information presents certain challenges for data processing, such as the "curse of dimensionality" leading to the "Hughes phenomenon", "strong correlation" due to high resolution, and "nonlinear characteristics" caused by varying surface reflectances. Consequently, dimensionality reduction of hyperspectral data emerges as a critical task. This paper begins by elucidating the principles and processes of hyperspectral image dimensionality reduction based on manifold theory and learning methods, in light of the nonlinear structures and features present in hyperspectral remote-sensing data, and formulates a dimensionality reduction process based on manifold learning. Subsequently, this study explores the capabilities of feature extraction and low-dimensional embedding for hyperspectral imagery using manifold learning approaches, including principal components analysis (PCA), multidimensional scaling (MDS), and linear discriminant analysis (LDA) for linear methods; and isometric mapping (Isomap), locally linear embedding (LLE), Laplacian eigenmaps (LE), Hessian locally linear embedding (HLLE), local tangent space alignment (LTSA), and maximum variance unfolding (MVU) for nonlinear methods, based on the Indian Pines hyperspectral dataset and Pavia University dataset. Furthermore, the paper investigates the optimal neighborhood computation time and overall algorithm runtime for feature extraction in hyperspectral imagery, varying by the choice of neighborhood k and intrinsic dimensionality d values across different manifold learning methods. Based on the outcomes of feature extraction, the study examines the classification experiments of various manifold learning methods, comparing and analyzing the variations in classification accuracy and Kappa coefficient with different selections of neighborhood k and intrinsic dimensionality d values. Building on this, the impact of selecting different bandwidths t for the Gaussian kernel in the LE method and different Lagrange multipliers λ for the MVU method on classification accuracy, given varying choices of neighborhood k and intrinsic dimensionality d, is explored. Through these experiments, the paper investigates the capability and effectiveness of different manifold learning methods in feature extraction and dimensionality reduction within hyperspectral imagery, as influenced by the selection of neighborhood k and intrinsic dimensionality d values, identifying the optimal neighborhood k and intrinsic dimensionality d value for each method. A comparison of classification accuracies reveals that the LTSA method yields superior classification results compared to other manifold learning approaches. The study demonstrates the advantages of manifold learning methods in processing hyperspectral image data, providing an experimental reference for subsequent research on hyperspectral image dimensionality reduction using manifold learning methods.

Using only the red-edge bands is sufficient to detect tree stress: A case study on the early detection of PWD using hyperspectral drone images

Pine wilt disease (PWD) is a destructive disease of pine trees caused by the pine wood nematode and early identification is crucial for disease control. Hyperspectral drone imagery has the potential to achieve early detection, although specific methods have not been sufficiently explored, including the spectral characteristics of early infections and the most efficient identification methods. This study aimed to examine the spectral responses to early infection, quantify the separability of healthy and early infections, and compare the accuracy and efficiency of different methods including single bands, vegetation indices (VIs) and 1st derivative reflectance and indices. We collected hyperspectral drone data in southeast China and used linear discriminant analysis (LDA) to determine the separability of healthy trees and trees at an early stage of infection. We also used bands with the same wavelengths as Sentienl-2 images (denoted S2 bands) to propose a standard for band selection.We found that it was possible to separate healthy trees and those at an early stage of infection around 0.71–0.78 using different methods. Among individual bands, the red-edge bands had the highest separability of 0.74. Using standard vegetation indices resulted in separability between 0.67 and 0.71; in addition, we proposed three new indices that achieved separability between 0.73 and 0.75. The 1st derivative reflectance at 714 nm had the highest separability of 0.78 in this study, while using the 1st derivative reflectance indices was slightly less accurate. The classification accuracy was also slightly lower when using Random Forest (RF) with all bands, sensitive bands, and S2 bands.We conclude that, of the methods tested, the red-edge bands are most sensitive to early infection, and using the 1st derivative reflectance at 714 nm or using the 1st derivative reflectance at the red-edge and blue-edge inflection point (REIP and BEIP) was sufficient for early identification. Acquiring and using additional wavelengths hardly improved the classification. Using wavelengths similar to those in the Sentienl-2 images achieved similar results, and thus can be used as a standard for dimensionality reduction of hyperspectral data pertaining to forest disease. The newly proposed VIs and 1st derivative reflectances at the yellow-edge inflection point (YEIP) and BEIP delivered better performance than the other tested indices, and could be alternatives for early identification. This study proposes simple and practical methods for early identification of wilt and provides insights for efficient data acquisition and data reduction.

Inversion analysis of soil nitrogen content using hyperspectral images with different preprocessing methods

Unmanned aerial vehicle (UAV) hyperspectral data can provide accurate and detailed spectral information on the surface of features, leading to a better understanding of the physiological characteristics and optical properties of objects. Compared to satellite hyperspectral, UAV hyperspectral has a greater image spatial resolution, but it also collects more impurity information along with more image detail information. The nitrogen (N) content in shallow soils is of great significance for crop growth, and most of its remote sensing inversions are currently based on satellite images, whereas relatively few studies have used hyperspectral UAV images in the near-infrared band for inversions. Based on this, we propose a high-accuracy inversion model based on UAV hyperspectral imagery using an elastic net (EN) dimensionality reduction algorithm combined with a decision tree (DT) model. In this study, a hyperspectral imager carried by a drone was used to acquire the surface soil spectral data of winter wheat in the study area after sowing and fertilizing, and the measured N content of the sampled soil was determined by chemical analysis methods. We used various pre-processing methods to denoise and reduce the dimensionality of the original hyperspectral images and utilized a DT regression model to construct an inversion model of N content with different preprocessing results. Then we compared the accuracy of the models and used the optimal model to obtain the spatial distribution map of soil N content in the study area. The results show that (1) the dimensionality reduction of hyperspectral data can effectively remove redundant information and prevent data overfitting. Among them, the successive projections algorithm and correlation analysis method are more suitable for analyzing pre-processed spectral data, whereas the Lasso and EN algorithms are more suitable for processing original spectral data. (2) For the original spectral images, the EN dimensionality reduction algorithm (R2 of the EN-Unt-DT inversion model is 0.930, RMSE is 0.045) performed better than the other three algorithms. For the pre-processed spectral images, the successive projections algorithm (R2 of the SPA-Pre-DT inversion model is 0.906, RMSE is 0.055) performed better than the other three algorithms. The constructed EN-Unt-DT inversion model had the highest fitting accuracy. The research results provide a corresponding technical reference for the inversion of soil N content using UAV NIR hyperspectral data.

An Intelligent Sorting Method of Film in Cotton Combining Hyperspectral Imaging and the AlexNet-PCA Algorithm.

Long-staple cotton from Xinjiang is renowned for its exceptional quality. However, it is susceptible to contamination with plastic film during mechanical picking. To address the issue of tricky removal of film in seed cotton, a technique based on hyperspectral images and AlexNet-PCA is proposed to identify the colorless and transparent film of the seed cotton. The method consists of black and white correction of hyperspectral images, dimensionality reduction of hyperspectral data, and training and testing of convolutional neural network (CNN) models. The key technique is to find the optimal way to reduce the dimensionality of the hyperspectral data, thus reducing the computational cost. The biggest innovation of the paper is the combination of CNNs and dimensionality reduction methods to achieve high-precision intelligent recognition of transparent plastic films. Experiments with three dimensionality reduction methods and three CNN architectures are conducted to seek the optimal model for plastic film recognition. The results demonstrate that AlexNet-PCA-12 achieves the highest recognition accuracy and cost performance in dimensionality reduction. In the practical application sorting tests, the method proposed in this paper achieved a 97.02% removal rate of plastic film, which provides a modern theoretical model and effective method for high-precision identification of heteropolymers in seed cotton.

Full-surface defect detection of navel orange based on hyperspectral online sorting technology.

The whole-surface hyperspectral image acquisition of navel orange is particularly important for surface defect detection and quality classification. Because the light intensity at the edge of the navel orange is lower than that in the middle, the defects on the surface of the navel orange cannot be effectively identified. In this paper, a hyperspectral online sorting device for the whole-surface defects of navel orange is proposed. First of all, the image data of navel orange is collected by online detection sorting equipment and the spectral image of the characteristic wave peak of 1655.72nm was extracted. Then, the light intensity at the edge of the navel orange is enhanced by nonuniformity correction based on quadratic curve fitting, and the light intensity correction of the navel orange is realized. Finally, the corrected image is segmented by the threshold to obtain surface defects, and the number of surface defect pixels is improved effectively compared with that before light intensity correction. Ultimately, the online sorting test is carried out, and the detection accuracy is 100%. This indicates that this method effectively improves the sensitivity of defect detection. At the same time, the dimensionality reduction of hyperspectral data is also carried out, which is conducive to improving the efficiency of online detection.

Double Deep Q-Network for Hyperspectral Image Band Selection in Land Cover Classification Applications

Hyperspectral data usually consists of hundreds of narrow spectral bands and provides more detailed spectral characteristics compared to commonly used multispectral data in remote sensing applications. However, highly correlated spectral bands in hyperspectral data lead to computational complexity, which limits many applications or traditional methods when applied to hyperspectral data. The dimensionality reduction of hyperspectral data becomes one of the most important pre-processing steps in hyperspectral data analysis. Recently, deep reinforcement learning (DRL) has been introduced to hyperspectral data band selection (BS); however, the current DRL methods for hyperspectral data BS simply remove redundant bands, lack the significance analysis for the selected bands, and the reward mechanisms used in DRL only take basic forms in general. In this paper, a new reward mechanism strategy has been proposed, and Double Deep Q-Network (DDQN) is introduced during BS using DRL to improve the network stabilities and avoid local optimum. To verify the effect of the proposed BS method, land cover classification experiments were designed and carried out to analyze and compare the proposed method with other BS methods. In the land cover classification experiments, the overall accuracy (OA) of the proposed method can reach 98.37%, the average accuracy (AA) is 95.63%, the kappa coefficient (Kappa) is 97.87%. Overall, the proposed method is superior to other BS methods. Experiments have also shown that the proposed method works not only for airborne hyperspectral data (AVIRIS and HYDICE), but also for hyperspectral satellite data, such as PRISMA data. When hyperspectral data is applied to similar applications, the proposed BS method could be a candidate for the BS preprocessing options.

Two New Indices for Unsupervised Dimensionality Reduction of Hyperspectral Data

Hyperspectral imaging, with hundreds of spectral bands provide a rich data source for thematic mapping. However, the high dimensionality of these images is a challenge which is commonly treated by dimensionality reduction techniques. In this paper, a pixel-based hyperspectral dimensionality reduction technique is proposed in which the spectral signature curve (SSC) is divided into segments with equal lengths. Afterwards two new mathematical indices, which are based on integral and L2-norm, are proposed to transform each SSC segment into a new feature of the reduced feature vector. The proposed method considers the ordinance of SSC and, unlike PCA, can be applied individually to all pixels independently and simultaneously which means the method is applicable in parallel processing. The proposed data reduction technique was applied to two well-known agricultural hyperspectral scenes and was compared to wavelet and PCA data reduction techniques. The obtained results proved the efficiency of the proposed method from both classification accuracy and processing time aspects.

Intelligent identification of film on cotton based on hyperspectral imaging and convolutional neural network.

The identification of the film on cotton is of great significance for the improvement of cotton quality. Most of the existing technologies are dedicated to removing colored foreign fibers from cotton using photoelectric sorting methods. However, the current technologies are difficult to identify colorless transparent film, which becomes an obstacle for the harvest of high-quality cotton. In this paper, an intelligent identification method is proposed to identify the colorless and transparent film on cotton, based on short-wave near-infrared hyperspectral imaging and convolutional neural network (CNN). The algorithm includes black-and-white correction of hyperspectral images, hyperspectral data dimensionality reduction, CNN model training and testing. The key technology is that the features of the hyperspectral image data are degraded by the principal component analysis (PCA) to reduce the amount of computing time. The main innovation is that the colorless and transparent film on cotton can be accurately identified through a CNN with the performance of automatic feature extraction. The experimental results show that the proposed method can greatly improve the identification precision, compared with the traditional methods. After the simulation experiment, the method proposed in this paper has a recognition rate of 98.5% for film. After field testing, the selection rate of film is as high as 96.5%, which meets the actual production needs.

Assessment of vegetation conservation status in plateau areas based on multi‐view and difference identification

SummaryVegetation is the center connecting the atmosphere, hydrosphere and biosphere, and it plays an irreplaceable and important role in maintaining the regional ecological balance and affecting the changes of the global ecological environment. Therefore, it is of great practical significance to deeply study the dynamic changes of regional vegetation, predict its future development trend, and comprehensively evaluate its ecological security status. Based on this, this paper combines the multi‐view and difference identification methods to carry out an evaluation study of the vegetation protection status in the plateau area. This paper takes the Huangshui watershed in Qinghai Province as the research area, uses HJ‐1 A star HSI hyperspectral remote sensing images, field measured vegetation canopy spectrum and other auxiliary classification data as data sources, and uses emerging hyperspectral remote sensing technology for analysis. After the above data is preprocessed, spectral data form transformation, hyperspectral data dimensionality reduction and image multi‐view classification feature selection, we use the CART decision tree to classify typical vegetation remote sensing. Finally, we use the landscape ecological security evaluation model to evaluate the vegetation ecological security of the basin. The simulation test verifies the applicability of the scheme in the classification of vegetation hyperspectral remote sensing in complex terrain areas.

Dimensionality reduction and classification for hyperspectral image based on robust supervised ISOMAP

ABSTRACT This study contributes to classification and recognition of hyperspectral image in industrial sorting, mineral identification, geological exploration, environmental survey, and other industrial production fields. The finding eliminates the redundancy of hyperspectral data and improves the classification accuracy of hyperspectral image. Hyperspectral remote sensing data are inherently nonlinear, and traditional linear dimensionality reduction methods cannot effectively reveal the nonlinear structure contained in the data. Manifold learning is a nonlinear dimensionality reduction method which has been developed rapidly in recent years. The intrinsic structure of the data is preserved by Isometric Feature Mapping (ISOMAP), but it is sensitive to noise and is not conducive to final classification. Prior studies neglected a higher dimensional manifold real structure in hyperspectral data dimensionality reduction due to lack of consideration credibility information and class information of samples in calculating manifold distance. This study proposes a Robust Supervised ISOMAP method. Data sample credibility mode is constructed, and a data similarity measure named triple geodesic distance is defined by introducing samples’ credibility information, class information, and neighborhood information. Triple geodesic distance fitting is proven. Multidimensional scaling analysis (MDS) and generalized regression neural network are used to construct the embedded coordinates of training samples and test samples. The results show that the proposed method has higher anti-noise performance than the traditional one. Also, the runtime is feasible, and can effectively improve the classification precision of the hyperspectral images, in comparison to traditional ISOMAP methods. The conclusion is that Robust Supervised ISOMAP is an effective nonlinear dimensionality reduction method for hyperspectral remote sensing image.

Supervised and Unsupervised Clustering Based Dimensionality Reduction of Hyperspectral Data

Nowadays, hyperspectral images (HIs) are widely used for land cover land use (LCLU) mapping. Hyperspectral sensors collect spectral data in numerous adjacent spectral bands, which are usually redundant. Hyperspectral data processing comes with important challenges such as huge processing time, difficulties in transfer, and storage. In this study, two supervised and unsupervised dimensionality reduction methods are proposed for hyperspectral feature extraction based on the band clustering technique. In the first method, the unsupervised method, after the unsupervised band clustering stage with some statistical attributes, the principal component transform is used in each cluster, and the first PC component is considered an extracted feature. In the second method, the supervised method, bands are clustered based on training samples mean vectors of each class, and the weighted mean operator is used for feature extraction in each cluster. The experiment is conducted on the classification of real famous HI named Indian Pines. Comparing the obtained results and some other state of art methods proved the proposed method's efficiency.

Folded LDA: Extending the Linear Discriminant Analysis Algorithm for Feature Extraction and Data Reduction in Hyperspectral Remote Sensing

The rich spectral information provided by hyperspectral imaging (HSI) has made this technology very useful in the classification of remotely sensed data. However, classification of hyperspectral data is typically affected by noise and the Hughes phenomenon due to the presence of hundreds of spectral bands and correlation among them, with usually a limited number of samples for training. Linear Discriminant Analysis (LDA) is a well-known technique that has been widely used for supervised dimensionality reduction of hyperspectral data. However, the use of LDA in hyperspectral remote sensing is limited due to 1) its poor performance on small training datasets and 2) the limited number of features that can be selected i.e. c-1 where c is the number of classes in the data. To solve these problems, this work presents a Folded LDA (F-LDA) for dimensionality reduction of remotely sensed HSI data in Small Sample Size (SSS) scenarios. The proposed approach allows many more discriminant features to be selected in comparison to the conventional LDA since the selection is no longer bound by the limiting factor, leading to significantly higher accuracy in the classification of pixels under SSS restrictions. The proposed approach is evaluated on five different datasets, where the experimental results demonstrate the superiority of the F-LDA to the conventional LDA in terms of not only higher classification accuracy but also reduced computational complexity, and reduced contiguous memory requirements.

Nonlinear dimensionality reduction of hyperspectral data based on spectral information divergence preserving principle

The paper proposes a nonlinear dimensionality reduction technique for hyperspectral data based on the principle of preserving the spectral information divergence (SID). In this technique, the spectral information divergence is used as a spectral dissimilarity measure in both the original and reduced space. In the experimental studies performed using open hyperspectral images, we solve the problem of per-pixel classification of hyperspectral images. The developed method extends the existing hyperspectral data analysis toolkit, which allows reducing the dimensionality of hyperspectral data based on the principle of preserving specified spectral dissimilarity measures.

Spectral transformation based on nonlinear principal component analysis for dimensionality reduction of hyperspectral images

ABSTRACTManaging transmission and storage of hyperspectral (HS) images can be extremely difficult. Thus, the dimensionality reduction of HS data becomes necessary. Among several dimensionality reduction techniques, transform-based have found to be effective for HS data. While spatial transformation techniques provide good compression rates, the choice of the spectral decorrelation approaches can have strong impact on the quality of the compressed image. Since HS images are highly correlated within each spectral band and in particular across neighboring bands, the choice of a decorrelation method allowing to retain as much information content as possible is desirable. From this point of view, several methods based on PCA and Wavelet have been presented in the literature. In this paper, we propose the use of NLPCA transform as a method to reduce the spectral dimensionality of HS data. NLPCA represents in a lower dimensional space the same information content with less features than PCA. In these terms, aim of this research is focused on the analysis of the results obtained through the spectral decorrelation phase rather than taking advantage of both spectral and spatial compression. Experimental results assessing the advantage of NLPCA with respect to standard PCA are presented on four different HS datasets.

Marginal discriminant analysis using support vectors for dimensionality reduction of hyperspectral data

ABSTRACTFeature extraction (FE) is an efficient pre-processing step in hyperspectral image (HSI) classification. This article proposes a novel supervised FE method based on graph embedding framework (GEF). This method, which is called marginal discriminant analysis using support vectors (MDSV), can be used as a linear dimensionality reduction approach. The proposed method constructs inner and support graphs to capture both global and local structures of data points. The global geometrical structure of data in each class is described by the inner graph. The support graph uses support vectors (SVs) to detect the local inter-class structure of different classes. Incorporating these graphs enables MDSV to maximize the margin between classes in the projected space. Implementation of MDSV on four benchmark hyperspectral datasets confirms its efficiency as an appropriate pre-processing method before classification of HSIs.

Parallel and Distributed Dimensionality Reduction of Hyperspectral Data on Cloud Computing Architectures

Cloud computing offers the possibility to store and process massive amounts of remotely sensed hyperspectral data in a distributed way. Dimensionality reduction is an important task in hyperspectral imaging, as hyperspectral data often contains redundancy that can be removed prior to analysis of the data in repositories. In this regard, the development of dimensionality reduction techniques in cloud computing environments can provide both efficient storage and preprocessing of the data. In this paper, we develop a parallel and distributed implementation of a widely used technique for hyperspectral dimensionality reduction: principal component analysis (PCA), based on cloud computing architectures. Our implementation utilizes Hadoop's distributed file system (HDFS) to realize distributed storage, uses Apache Spark as the computing engine, and is developed based on the map-reduce parallel model, taking full advantage of the high throughput access and high performance distributed computing capabilities of cloud computing environments. We first optimized the traditional PCA algorithm to be well suited for parallel and distributed computing, and then we implemented it on a real cloud computing architecture. Our experimental results, conducted using several hyperspectral datasets, reveal very high performance for the proposed distributed parallel method.

Target oriented dimensionality reduction of hyperspectral data by Kernel Fukunaga–Koontz Transform

Principal component analysis (PCA) is a popular technique in remote sensing for dimensionality reduction. While PCA is suitable for data compression, it is not necessarily an optimal technique for feature extraction, particularly when the features are exploited in supervised learning applications (Cheriyadat and Bruce, 2003) [1]. Preserving features belonging to the target is very crucial to the performance of target detection/recognition techniques. Fukunaga–Koontz Transform (FKT) based supervised band reduction technique can be used to provide this requirement. FKT achieves feature selection by transforming into a new space in where feature classes have complimentary eigenvectors. Analysis of these eigenvectors under two classes, target and background clutter, can be utilized for target oriented band reduction since each basis functions best represent target class while carrying least information of the background class. By selecting few eigenvectors which are the most relevant to the target class, dimension of hyperspectral data can be reduced and thus, it presents significant advantages for near real time target detection applications. The nonlinear properties of the data can be extracted by kernel approach which provides better target features. Thus, we propose constructing kernel FKT (KFKT) to present target oriented band reduction. The performance of the proposed KFKT based target oriented dimensionality reduction algorithm has been tested employing two real-world hyperspectral data and results have been reported consequently.

Spectral–Spatial Classification of Hyperspectral Data Using 3-D Morphological Profile

A new spectral–spatial method based on a 3-D morphological profile (3D-MP) is proposed for hyperspectral data classification. As an extension of a previous approach, the proposed method uses both the spectral and spatial information for classification. First, random projection (RP) is used for dimensionality reduction of hyperspectral data. After RP in spectral domain, a novel 3D-MP method is proposed to exploit the dependence between data. Finally, the classification is performed by the widely used support vector machine classifier. Our experiments reveal that the proposed approach exploits the 3-D spectral–spatial feature to provide the state-of-the-art classification results for different hyperspectral data sets.

Optimal band selection for hyperspectral data with improved differential evolution

The optimal band selection has been a hot research topic and one of the difficult problems in the field of hyperspectral remote sensing. To address the issue of hyperspectral data and improve the data processing speed, we propose a new method to improve differential evolution algorithm. We use the ENVI software on the pretreatment of original spectral data and dividing subspace. The experiments are also conducted with the improved DE algorithm. In experiments, different evaluation criteria of band selection are taken as the fitness function value of the DE algorithm. By calculating the optimal band combination in different dimensions of size, we can obtain the optimal selection band of hyperspectral data in this paper. Experiments show that the improved differential evolution algorithm for hyperspectral data dimensionality reduction model not only obtains the optimal band combination, but also greatly improves the classification accuracy.

Discriminative Spectral–Spatial Margin-Based Semisupervised Dimensionality Reduction of Hyperspectral Data

The past few years have witnessed prosperity of spectral-spatial processing of hyperspectral images. In this letter, in order to determine the optimal projection subspace of spectrums, we define discriminate spectral-spatial margins (DSSMs) to reveal the local information of hyperspectral pixels and explore the global structures of both labeled and unlabeled data via low-rank representation (LRR). Heterogeneous and homogeneous spectral-spatial neighbors of hyperspectral pixels are used to define DSSMs. By maximizing the DSSM of hyperspectral data and casting an LRR manifold regularizer on finding better projection, both the local and global information of hyperspectral data can be well explored to determine more discriminative features. Some experiments are taken on several real hyperspectral data sets, and the results exhibit its efficiency and superiority to the counterparts, when only a small number of labeled samples are available.