Supervised Feature Extraction Techniques Research Articles

Feature extraction of small underwater sonar targets using neighborhood discriminant analysis

Linear discriminant analysis (LDA) stands as a widely used supervised feature extraction technique that maps data onto a low-dimensional subspace such that the between-class scatter is maximized while within-class scatter is minimized. Despite its utility, LDA faces many challenges, particularly when the within-class scatter matrix becomes singular due to small sample size. Other dimensionality reduction techniques, such as Neighbourhood Component Analysis (NCA), have been proposed as alternatives to LDA. NCA learns a linear transformation that maximizes the likelihood that datapoints of the same class are clustered together in the lower-dimensional space. However, the optimization of NCA relies on a non-convex cost function, making it prone to local minima. To address the challenges faced by both LDA and NCA, we propose a novel dimensionality reduction method named neighborhood discriminant analysis (NDA). Like NCA, NDA learns a linear transformation that aims to cluster datapoints based on class label. However, NDA is framed as an eigendecomposition problem, eliminating the need for non-convex optimization. We demonstrate the new approach on real small target sonar data. [Work funded by the ONR grant numbers N000142112420 and N000142312503, and DoD Navy (NEEC) Grant No. N001742010016.]

Deep fisher autoencoder combined with self-organizing map for visual industrial process monitoring

Feature extraction plays an important role in industrial process monitoring. Autoencoder and its deep framework, deep autoencoder, are used to extract latent features from complex data. However, the latent features extracted by autoencoder through unsupervised learning may not be useful for discriminative tasks. Fisher discriminant analysis (FDA) is another widely used supervised feature extraction technique that take full advantage of the Fisher criterion to enable the extracted discriminative features to maximize inter-class distance while minimizing intra-class distance. Drawing on FDA and autoencoder, this study proposes Fisher autoencoder (FAE) to extract discriminative features. FAE uses the Fisher criterion to guide the autoencoder in minimizing the reconstruction error while enabling the extracted features by the hidden layer to increase the separation between classes. We stack FAE to derive deep FAE (DFAE) for feature extraction, then we combine DFAE with self-organizing map (DFAE-SOM), which is a tool typically used in visualization for visual process monitoring. Tennessee Eastman process and an actual dataset of the blade icing of wind turbine are applied to test the performance of DFAE-SOM. The experiment demonstrates that DFAE increases the separation between classes more than DAE and other standard techniques. Therefore, DFAE is conducive to visualization and improves the accuracy of process monitoring.

Class-adapted local fisher discriminant analysis to reduce highly-dimensioned data on commodity hardware: application to hyperspectral images

Local Fisher Discriminant Analysis (LFDA) is a supervised feature extraction technique that proved to be efficient in reducing several types of data. However, it depends on the number of samples per class in a way that can lead, when classes are too large, to a consumption of all the memory of a commodity hardware, or to a disability to even run. To work around this limit, we hereby propose to introduce a parameter that adapts LFDA to the data’s classes while accounting for the available resources on the used machine. In fact, according to this parameter, LFDA will consider a larger class as a set of smaller sub-classes and will process these latter instead of the larger one. We are calling our proposed optimization the class-adapted LFDA, noted caLFDA. We also propose a Python implementation of LFDA and prove it more effective than the existent MATLAB implementation. To assess the efficiency of caLFDA, we applied it to reduce several hyperspectral images and compared the results of classifying the reduced images to the ones we get when using the original LFDA to reduce the data. When the hyperspectral images are too large for LFDA to be able to reduce them, we compare caLFDA’s results to the ones we get with the most commonly used Principle Component Analysis (PCA).

A novel supervised feature extraction algorithm: enhanced within-class linear discriminant analysis

Linear discriminant analysis (LDA) is one of the most popular supervised feature extraction techniques used in machine learning and pattern classification. However, LDA only captures global structure information of the data and ignores the structure information of local data points. In this paper, a novel supervised feature extraction algorithm called enhanced within-class linear discriminant analysis (EWLDA) is proposed. More specifically, we define a local within-class scatter matrix to model the local structure information provided by local data samples. In order to balance the tradeoff between global and local structure information, a tuning parameter is also introduced. Experimental results on two image databases demonstrate the effectiveness of our algorithm.

A novel supervised feature extraction algorithm: enhanced within-class linear discriminant analysis

Linear discriminant analysis (LDA) is one of the most popular supervised feature extraction techniques used in machine learning and pattern classification. However, LDA only captures global structure information of the data and ignores the structure information of local data points. In this paper, a novel supervised feature extraction algorithm called enhanced within-class linear discriminant analysis (EWLDA) is proposed. More specifically, we define a local within-class scatter matrix to model the local structure information provided by local data samples. In order to balance the tradeoff between global and local structure information, a tuning parameter is also introduced. Experimental results on two image databases demonstrate the effectiveness of our algorithm.

Noise Effects on Feature Mining Non-Parametric Supervised Feature Extraction Techniques

In this paper two famous and commonly used feature mining non-parametric supervised feature extraction techniques (NSFETs) called Non-parametric Weighted Feature Extraction (NWFE) and Decision Boundary Feature Extraction (DBFE) are analyzed to see their efficiency in the presence of noise. In particularly these feature extraction techniques are used in classification as they give better classification accuracy. This study reveals that NSFETs are very sensitive to noise because of which the number of features increases and we get low classification accuracy. In order to see the behavior of NSFETs, spatial and spectral information from hyperspectral image classification is used. The experimental results show that in the presence of noise, spectral information is much more effected than the spatial information when features are extracted using the NSFETs. It is also examined that NWFE is more affected by noise than DBFE. The linear filtering technique is used just before the classifier in order to mitigate the noise effects in NSFETs. Using linear filtering just before the classifier does improve the final classification accuracy but with high number of spatial and spectral features. This does not satisfy the one of the main purpose of feature extraction and that is feature reduction.

Land-cover classification using both hyperspectral and LiDAR data

The increased availability of data from different satellite and airborne sensors from a particular scene makes it desirable to jointly use data from multiple data sources for improved information extraction and classification. In particular, hyperspectral sensors provide valuable spectral information that can be used to discriminate different classes of interest, but they do not provide structural and elevation information. On the other hand, LiDAR data can extract useful information related to the size, structure and elevation of different objects, but cannot model the spectral characteristics of different materials. In this paper, a new classification framework is proposed by considering the integration of hyperspectral and LiDAR data. In this case, the recently introduced theoretically sound attribute profile (AP) is considered to model the spatial information of LiDAR and hyperspectral data. In parallel, in order to reduce the redundancy of the hyperspectral data and address the so-called curse of dimensionality, supervised feature extraction techniques are taken into account. Then, the new features obtained by the AP and the supervised feature extraction techniques are concatenated into a stacked vector. The final classification map is achieved by using either support vector machine or random forest classification techniques. The proposed method was applied on two data sets and the obtained results were compared in terms of classification accuracies and CPU processing time. From the results it can be concluded that the proposed method can classify the integration of hyperspectral and LiDAR data accurately in a very acceptable CPU processing time. It should be noted that the proposed method is fully automatic and there is no need to set any parameters to increase the favourability of the proposed method.

Global plus local: A complete framework for feature extraction and recognition

Linear discriminant analysis (LDA) is one of the most popular supervised feature extraction techniques used in machine learning and pattern classification. However, LDA only captures global geometrical structure information of the data and ignores the geometrical structure information of local data points. Though many articles have been published to address this issue, most of them are incomplete in the sense that only part of the local information is used. We show here that there are total three kinds of local information, namely, local similarity information, local intra-class pattern variation, and local inter-class pattern variation. We first propose a new method called enhanced within-class LDA (EWLDA) algorithm to incorporate the local similarity information, and then propose a complete framework called complete global–local LDA (CGLDA) algorithm to incorporate all these three kinds of local information. Experimental results on two image databases demonstrate the effectiveness of our algorithms.

Classification of hyperspectral data using extended attribute profiles based on supervised and unsupervised feature extraction techniques

The classification of remote sensing data based on the exploitation of spatial features extracted with morphological and attribute profiles has been recently gaining importance. With the development of efficient algorithms to construct the profiles for large datasets, such methods are becoming even more relevant. When dealing with hyperspectral imagery, the profiles are traditionally built on the first few principal components computed from the data. However, it needs to be determined if other feature reduction approaches are better suited to create base images for the profiles. In this article, we explore the use of profiles based on features derived from three supervised feature extraction techniques (i.e. Discriminant Analysis Feature Extraction, Decision Boundary Feature Extraction and Non-parametric Weighted Feature Extraction) and two unsupervised feature-extraction techniques (i.e. Principal Component Analysis (PCA) and Kernel PCA) in classification and compare the classification accuracies obtained by using different techniques for two different classification methods. The obtained results indicate significant improvements in the accuracies using the supervised feature extraction methods. However, the choice of the method affects the quality of the results for different datasets depending on the availability of the training samples.

Discriminant Independent Component Analysis

A conventional linear model based on Negentropy maximization extracts statistically independent latent variables which may not be optimal to give a discriminant model with good classification performance. In this paper, a single-stage linear semisupervised extraction of discriminative independent features is proposed. Discriminant independent component analysis (dICA) presents a framework of linearly projecting multivariate data to a lower dimension where the features are maximally discriminant with minimal redundancy. The optimization problem is formulated as the maximization of linear summation of Negentropy and weighted functional measure of classification. Motivated by independence among extracted features, Fisher linear discriminant is used as the functional measure of classification. Experimental results show improved classification performance when dICA features are used for recognition tasks in comparison to unsupervised (principal component analysis and ICA) and supervised feature extraction techniques like linear discriminant analysis (LDA), conditional ICA, and those based on information theoretic learning approaches. dICA features also give reduced data reconstruction error in comparison to LDA and ICA method based on Negentropy maximization.

Band Selection in Multispectral Images by Minimization of Dependent Information

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> In this paper, a band selection technique for hyperspectral image data is proposed. Supervised feature extraction techniques allow a reduction of the dimensionality to extract relevant features through a labeled training set. This implies an analysis of the existing class distributions, which usually means, in the case of hyperspectral imaging, a large number of samples, making the labeling process difficult. A possible alternative could be the use of information measures, which are the basis of the proposed method. The present approach basically behaves as an unsupervised feature selection criterion, to obtain the relevant spectral bands from a set of sample images. The relations of information content between spectral bands are analyzed, leading to the proposed technique based on the minimization of the dependent information between spectral bands, while trying to maximize the conditional entropies of the selected bands. </para>