Maximize Class Separability Research Articles

Sparse feature selection using hypergraph Laplacian-based semi-supervised discriminant analysis

Feature selection, as a dimension reduction technique in data mining and pattern recognition, aims to select the most discriminative features and improve the learning performance. With an abundance of unlabeled data readily available across various applications, semi-supervised feature selection has emerged as a promising approach. While most semi-supervised feature selection methods rely on simple graphs to preserve the geometrical structure of data, this approach often fails in capturing the high-order relationships present in many real-world applications. In contrast, hypergraphs offer the ability to encode more complex structures of data beyond what a simple graph can achieve. In this paper, we propose a feature selection method formulated in the trace ratio form, integrating hypergraph Laplacian-based semi-supervised discriminant analysis (SDA) and the mixed convex and non-convex ℓ2,p-norm (0<p≤1) regularization. The proposed trace ratio-based method, called HSDAFS, leverages the discriminative information from labeled data to maximize class separability while also utilizing the hypergraph Laplacian to capture the geometrical structure and high-order relationships within both labeled and unlabeled data. The ℓ2,p-norm regularization in the proposed HSDAFS provides improved sparsity over the ℓ2,1-norm. It ensures that the projection matrix is row-sparse, enabling the effective joint selection of discriminative features across all data. To solve the trace ratio-based HSDAFS method, we convert it into a trace difference method and propose an iterative algorithm. Experiments on several datasets demonstrate that HSDAFS is more effective in selecting the most discriminative features compared to other methods.

Distract Your Attention: Multi-Head Cross Attention Network for Facial Expression Recognition.

This paper presents a novel facial expression recognition network, called Distract your Attention Network (DAN). Our method is based on two key observations in biological visual perception. Firstly, multiple facial expression classes share inherently similar underlying facial appearance, and their differences could be subtle. Secondly, facial expressions simultaneously exhibit themselves through multiple facial regions, and for recognition, a holistic approach by encoding high-order interactions among local features is required. To address these issues, this work proposes DAN with three key components: Feature Clustering Network (FCN), Multi-head Attention Network (MAN), and Attention Fusion Network (AFN). Specifically, FCN extracts robust features by adopting a large-margin learning objective to maximize class separability. In addition, MAN instantiates a number of attention heads to simultaneously attend to multiple facial areas and build attention maps on these regions. Further, AFN distracts these attentions to multiple locations before fusing the feature maps to a comprehensive one. Extensive experiments on three public datasets (including AffectNet, RAF-DB, and SFEW 2.0) verified that the proposed method consistently achieves state-of-the-art facial expression recognition performance. The DAN code is publicly available.

ArcFace: Additive Angular Margin Loss for Deep Face Recognition.

Recently, a popular line of research in face recognition is adopting margins in the well-established softmax loss function to maximize class separability. In this paper, we first introduce an Additive Angular Margin Loss (ArcFace), which not only has a clear geometric interpretation but also significantly enhances the discriminative power. Since ArcFace is susceptible to the massive label noise, we further propose sub-center ArcFace, in which each class contains K sub-centers and training samples only need to be close to any of the K positive sub-centers. Sub-center ArcFace encourages one dominant sub-class that contains the majority of clean faces and non-dominant sub-classes that include hard or noisy faces. Based on this self-propelled isolation, we boost the performance through automatically purifying raw web faces under massive real-world noise. Besides discriminative feature embedding, we also explore the inverse problem, mapping feature vectors to face images. Without training any additional generator or discriminator, the pre-trained ArcFace model can generate identity-preserved face images for both subjects inside and outside the training data only by using the network gradient and Batch Normalization (BN) priors. Extensive experiments demonstrate that ArcFace can enhance the discriminative feature embedding as well as strengthen the generative face synthesis.

Uncorrelated Discriminant Projection Based on Maximum Margin Criterion and Its Kernelized Extension

A new linear dimensionality reduction algorithm called uncorrelated discriminant projection (UDP) is proposed in this paper. The proposed UDP algorithm is based on the maximum margin criterion (MMC) which aim at maximizing class separation after dimension reduction. By imposing an uncorrelated constraint in the objective function, UDP extracts statistically uncorrelated features which are important in many pattern recognition problems. Moreover, we propose performing UDP in reproducing kernel Hilbert space (RKHS) which leads to a nonlinear variant of UDP called kernel uncorrelated discriminant projections (KUDP). In order to demonstrate the effectiveness and efficiency of the newly proposed algorithms, we conducted experiments on two benchmark face databases. The experimental results indicates that both UDP and KUDP are able to find face subspaces optimal for recognition.

A Novel Approach to Fuzzy-Based Facial Feature Extraction and Face Recognition

Generalized two-dimensional Fisher’s linear discriminant (G-2DFLD) is an effective feature extraction technique that maximizes class separability along row and column directions simultaneously. In this paper, we propose a fuzzy logic-based feature extraction technique, called fuzzy generalized two-dimensional Fisher’s linear discriminant analysis (FG-2DLDA) method which is extended version of the G-2DFLD method. This paper also explores the use of the proposed method for face recognition using radial basis function (RBF) neural network as a classifier. Fuzzy membership matrix values are calculated by fuzzy k -nearest neighbour (F k- NN) algorithm for the training samples. These fuzzy membership values are combined with the training samples to generate global mean and class-wise mean training images. Thereafter, the global and class-wise mean images are used to generate fuzzy within- and between-class scatter matrices along the both directions. Finally, by solving the Eigen value problems of these scatter matrices, we find the optimal fuzzy projection vectors, which actually used to generate more discriminant features. Our proposed method has been evaluated on the four public face databases using RBF neural network and establish that the proposed FG-2DLDA method provides favourable recognition rates than some contemporary face recognition methods.

A Feature Selection Method based on the Pearson’s Correlation and Transformed Divergence Analysis

An improved feature selection method has been presented, which is based on Transformed Divergence (TD) considering weights of classes and Pearson’s correlation analysis. Using the improved method, this study evaluated several derived vegetation indices and texture measures based on Landsat-8 OLI data to determine their effect on improving the land cover classification separability in Jiangle county, Sanming city of Fujian province, China. The best vegetation indices combination was selected by the improved feature selection method and likewise, best textural combinations at different spatial resolution levels from multi-spectral bands or panchromatic band were obtained. The improved feature selection method found that a single feature could not maximize the separability of vegetation classes. When selecting four vegetation indices, the separability of vegetation classes can be maxmized significantly; and two textural measures were suitable for maxmizing the separability of vegetation classes. Overall, the result verifies that the feature selection method considering weights of classes and Pearson’s correlation coefficient can select optimal features to maximize class separability.

Efficient and effective strategies for cross-corpus acoustic emotion recognition

An important research direction in speech technology is robust cross-corpus and cross-language emotion recognition. In this paper, we propose computationally efficient and performance effective feature normalization strategies for the challenging task of cross-corpus acoustic emotion recognition. We particularly deploy a cascaded normalization approach, combining linear speaker level, nonlinear value level and feature vector level normalization to minimize speaker- and corpus-related effects as well as to maximize class separability with linear kernel classifiers. We use extreme learning machine classifiers on five corpora representing five languages from different families, namely Danish, English, German, Russian and Turkish. Using a standard set of suprasegmental features, the proposed normalization strategies show superior performance compared to benchmark normalization approaches commonly used in the literature.

A Two-Stage Optimization Strategy for Fuzzy Object-Based Analysis Using Airborne LiDAR and High-Resolution Orthophotos for Urban Road Extraction

In the last decade, object-based image analysis (OBIA) has been extensively recognized as an effective classification method for very high spatial resolution images or integrated data from different sources. In this study, a two-stage optimization strategy for fuzzy object-based analysis using airborne LiDAR was proposed for urban road extraction. The method optimizes the two basic steps of OBIA, namely, segmentation and classification, to realize accurate land cover mapping and urban road extraction. This objective was achieved by selecting the optimum scale parameter to maximize class separability and the optimum shape and compactness parameters to optimize the final image segments. Class separability was maximized using the Bhattacharyya distance algorithm, whereas image segmentation was optimized using the Taguchi method. The proposed fuzzy rules were created based on integrated data and expert knowledge. Spectral, spatial, and texture features were used under fuzzy rules by implementing the particle swarm optimization technique. The proposed fuzzy rules were easy to implement and were transferable to other areas. An overall accuracy of 82% and a kappa index of agreement (KIA) of 0.79 were achieved on the studied area when results were compared with reference objects created via manual digitization in a geographic information system. The accuracy of road extraction using the developed fuzzy rules was 0.76 (producer), 0.85 (user), and 0.72 (KIA). Meanwhile, overall accuracy was decreased by approximately 6% when the rules were applied on a test site. A KIA of 0.70 was achieved on the test site using the same rules without any changes. The accuracy of the extracted urban roads from the test site was 0.72 (KIA), which decreased to approximately 0.16. Spatial information (i.e., elongation) and intensity from LiDAR were the most interesting properties for urban road extraction. The proposed method can be applied to a wide range of real applications through remote sensing by transferring object-based rules to other areas using optimization techniques.

Multi-class Fukunaga Koontz discriminant analysis for enhanced face recognition

Linear subspace learning methods such as Fisher׳s Linear Discriminant Analysis (LDA), Unsupervised Discriminant Projection (UDP), and Locality Preserving Projections (LPP) have been widely used in face recognition applications as a tool to capture low dimensional discriminant information. However, when these methods are applied in the context of face recognition, they often encounter the small-sample-size problem. In order to overcome this problem, a separate Principal Component Analysis (PCA) step is usually adopted to reduce the dimensionality of the data. However, such a step may discard dimensions that contain important discriminative information that can aid classification performance. In this work, we propose a new idea which we named Multi-class Fukunaga Koontz Discriminant Analysis (FKDA) by incorporating the Fukunaga Koontz Transform within the optimization for maximizing class separation criteria in LDA, UDP, and LPP. In contrast to traditional LDA, UDP, and LPP, our approach can work with very high dimensional data as input, without requiring a separate dimensionality reduction step to make the scatter matrices full rank. In addition, the FKDA formulation seeks optimal projection direction vectors that are orthogonal which the existing methods cannot guarantee, and it has the capability of finding the exact solutions to the “trace ratio” objective in discriminant analysis problems while traditional methods can only deal with a relaxed and inexact “ratio trace” objective. We have shown using six face database, in the context of large scale unconstrained face recognition, face recognition with occlusions, and illumination invariant face recognition, under “closed set”, “semi-open set”, and “open set” recognition scenarios, that our proposed FKDA significantly outperforms traditional linear discriminant subspace learning methods as well as five other competing algorithms.

Regularized complete linear discriminant analysis

Linear discriminant analysis (LDA) searches for a linear transformation that maximizes class separability in a reduced dimensional space. Because LDA requires the within-class scatter matrix to be non-singular, it cannot be directly applied to small sample size (SSS) problems in which the number of available training samples is smaller than the dimensionality of the sample space. To solve SSS problems, this paper develops a system of regularized complete linear discriminant analysis (RCLDA). In RCLDA, two regularized criteria are used to derive discriminant vectors that include “regular” and “irregular” discriminant vectors in the range space and null space, respectively, of the within-class scatter matrix. Extensive experiments on the SSS problem of image recognition are performed to evaluate the proposed algorithm in terms of classification accuracy, and the experimental results demonstrate its effectiveness.

Game Theoretic Pattern Analysis for Identification of Odors/Gases Using Response of a Poorly Selective Sensor Array

This paper presents a novel approach toward the identification of odors/gases using game theory for feature selection. A previously reported tin-oxide-based sensor array capable of sensing at room temperature is chosen to extract the data. The sensor array is exposed to four different odors/gases and the response pattern obtained reveals its poor selectivity. The task of classifying the sampled data into four classes is modeled as a coalitional game in which each sensor of the array acted like a player forming coalitions with other players. A pay-off function is associated with each possible coalition of players with higher pay-offs being given to coalitions that maximize class separability of the data. Shapley value is used to quantify the contribution of each player yielding a standard pattern for each odor class. Each sample of the test vector is then matched with the standard pattern and the odor class yielding minimum Euclidean distance is assigned to the test sample. A weighting scheme for relative scaling of the test samples is also proposed. It is observed that more than 89% of the samples were identified correctly using the proposed technique thereby proving its efficacy.

Face recognition by generalized two-dimensional FLD method and multi-class support vector machines

This paper presents a novel scheme for feature extraction, namely, the generalized two-dimensional Fisher's linear discriminant (G-2DFLD) method and its use for face recognition using multi-class support vector machines as classifier. The G-2DFLD method is an extension of the 2DFLD method for feature extraction. Like 2DFLD method, G-2DFLD method is also based on the original 2D image matrix. However, unlike 2DFLD method, which maximizes class separability either from row or column direction, the G-2DFLD method maximizes class separability from both the row and column directions simultaneously. To realize this, two alternative Fisher's criteria have been defined corresponding to row and column-wise projection directions. Unlike 2DFLD method, the principal components extracted from an image matrix in G-2DFLD method are scalars; yielding much smaller image feature matrix. The proposed G-2DFLD method was evaluated on two popular face recognition databases, the AT&T (formerly ORL) and the UMIST face databases. The experimental results using different experimental strategies show that the new G-2DFLD scheme outperforms the PCA, 2DPCA, FLD and 2DFLD schemes, not only in terms of computation times, but also for the task of face recognition using multi-class support vector machines (SVM) as classifier. The proposed method also outperforms some of the neural networks and other SVM-based methods for face recognition reported in the literature.

A Generic Multi-dimensional Feature Extraction Method Using Multiobjective Genetic Programming

In this paper, we present a generic feature extraction method for pattern classification using multiobjective genetic programming. This not only evolves the (near-)optimal set of mappings from a pattern space to a multi-dimensional decision space, but also simultaneously optimizes the dimensionality of that decision space. The presented framework evolves vector-to-vector feature extractors that maximize class separability. We demonstrate the efficacy of our approach by making statistically-founded comparisons with a wide variety of established classifier paradigms over a range of datasets and find that for most of the pairwise comparisons, our evolutionary method delivers statistically smaller misclassification errors. At very worst, our method displays no statistical difference in a few pairwise comparisons with established classifier/dataset combinations; crucially, none of the misclassification results produced by our method is worse than any comparator classifier. Although principally focused on feature extraction, feature selection is also performed as an implicit side effect; we show that both feature extraction and selection are important to the success of our technique. The presented method has the practical consequence of obviating the need to exhaustively evaluate a large family of conventional classifiers when faced with a new pattern recognition problem in order to attain a good classification accuracy.

Automatic event picking in prestack migrated gathers using a probabilistic neural network

We describe algorithms for automating the process of picking seismic events in prestack migrated common depth image gathers. The approach uses supervised learning and statistical classification algorithms along with advanced signal/image processing algorithms. No model assumption is made, such as hyperbolic moveout. We train a probabilistic neural network for voxel classification using event times, subsurface points, and offsets (ground truth information) picked manually by expert interpreters. The key to success is using effective features that capture the important behavior of the measured signals. We test a variety of features calculated in a local neighborhood about the voxel under analysis. Selection algorithms ensure that we use only the features that maximize class separability. This event‐picking algorithm has the potential to reduce significantly the cycle time and cost of 3‐D prestack depth migration while making the velocity model inversion more robust.

A maximal predictive classification of Klebsielleae and of the yeasts.

The concepts of the numerical method of maximal predictive classification are illustrated with classifications of 13 species of enterobacteria and of 434 species of yeast. The method seeks to classify into a specified number of classes (k) such that more correct statements can be made about the constituent members than with any other classification. The best choice of k relates to the separation of the classes as measured by the average number of correct statements made for an individual assigned to a class to which it does not belong. The maximal predictive classifications are compared with previous classifications of the two groups, which seem to be poor predictively (in terms of the characters considered in this study). The results suggest that taxonomists may be more concerned with maximizing class separation rather than with prediction, but many more groups of organisms would need similar study before this view could be held with confidence.