Two-dimensional Linear Discriminant Analysis Research Articles

Bilateral two-dimensional linear discriminant analysis and its applications

Bilateral two-dimensional linear discriminant analysis and its applications

Improving two-dimensional linear discriminant analysis with L1 norm for optimizing EEG signal

Dimensionality reduction is a critical factor in processing high-dimensional datasets. The L1 norm-based Two-Dimensional Linear Discriminant Analysis (L1-2DLDA) is widely used for this purpose, but it remains sensitive to outliers and classes with large deviations, which deteriorates its performance. To address this limitation, the present study proposed Pairwise Sample Distance Two-Dimensional Linear Discriminant Analysis (PSD2DLDA), a novel method that modeled L1-2DLDA using pair-wise sample distances. To improve computational effectiveness, this study also introduced a streamlined variant, Pairwise Class Mean Distance Two-Dimensional Linear Discriminant Analysis (PCD2DLDA), which was based on distances between class mean pairs. Different from previous studies, this study utilized the projected sub-gradient method to optimize these two improved methods. Meanwhile, this study explored the interrelationship, limitations, and applicability of these two improved methods. The comparative experimental results on three datasets validated the outstanding performance of PSD2DLDA and PCD2DLDA methods. In particular, PSD2DLDA exhibited superior robustness compared to PCD2DLDA. Furthermore, applying these two methods to optimize electroencephalogram (EEG) signals effectively enhanced the decoding accuracy of motor imagery neural patterns, which offered a promising strategy for optimizing EEG signals processing in brain-computer interface (BCI) applications.

Discrimination of volatile organic compounds using a sensor array via a rapid method based on linear discriminant analysis

We investigated a discriminant method for volatile organic compounds (VOCs) in exhaled breath under contaminant air as well as purified air conditions using an array of multiple semiconductive gas sensors and the linear discriminant analysis (LDA) space. We analyzed the sensor signals of resistance changes employing our developed discriminant method in the LDA space. In particular, the sensor resistances at intervals at which a resistance change occurred owing to target gas introduction were used as test data. The linear discriminant (LD) scores from the test data in the two-dimensional (2-D) LDA space gradually shifted in the direction of the correct LD scores of the training data. Therefore, the target gases were discriminated using the angle of locus of the test data in the 2-D LDA space. Our results proved the viability of the proposed method to discriminate target gases under contaminant air conditions.

Natural Feature Recognition of Multi Pose Face Images based on Augmented Reality

Traditional face feature recognition methods have a few problems, such as low accuracy of image feature recognition and high time consumption of image feature recognition. This paper introduces augmented reality technology to realize natural feature recognition of multi pose face images. The multi pose features of face image are obtained by augmented reality technology, the color normalization of face image is realized by gamma algorithm, and the natural features of multi pose face image are extracted by three-dimensional registration technology. The feature error reconstruction of multi pose face image is realized according to sparse representation, the face image samples are projected into the optimal discriminant vector space by using two-dimensional linear discriminant analysis algorithm, and the natural feature recognition of multi pose face image is completed according to augmented reality technology. The experimental results show that the comprehensive accuracy of facial image expression recognition is as high as 95%, and the average error rate of natural feature recognition is only 3%; The time taken for feature recognition is only 3.5s, which shows that the method used in this study has higher efficiency of natural feature recognition of face image.

F $F$‐norm two‐dimensional linear discriminant analysis and its application on face recognition

Two-dimensional linear discriminant analysis (2DLDA) is a widely applied extension of LDA that can cope with matrix input samples directly. However, its construction is based on a squared F $F$ -norm which will lead to sensitivity to noise and outliers. In this paper, a square-free F $F$ -norm 2DLDA is proposed to improve the robustness of 2DLDA. By losing the squared operation, the proposed method weakens the influence of outliers and noise and at the same time keeps the geometric structure of data. It can be solved through an effective nongreedy iterative algorithm, with each subproblem having a closed-form solution. The algorithm is further proved to be convergent. Experiments on several human face image databases demonstrate the effectiveness and robustness of the proposed method.

Joint diagonalization for a pair of Hermitian quaternion matrices and applications to color face recognition

A new joint diagonalization algorithm for a pair of Hermitian quaternion matrices is derived incorporating real structure-preserving strategy. The structure-preserving joint diagonalization algorithm leads to a novel two-dimensional quaternion linear discriminant analysis (2D-QLDA) method for color face recognition and image reconstruction. 2D-QLDA is mathematically characterized by Hermitian quaternion generalized eigenvalue problem. A weighted norm is obtained as a new measurement to determine the distances among Fisher feature matrices, which helps us avoid generating projected images explicitly. Numerical results based on the real face databases indicate that 2D-QLDA performs better than other 2D-LDA-like methods in color face recognition and is effective in image reconstruction.

Integration of Two-Dimensional Kernel Principal Component Analysis Plus Two-Dimensional Linear Discriminant Analysis with Convolutional Neural Network for Finger Vein Recognition

High complexity and low recognition rate are two common problems with the current finger vein recognition methods. To solve these problems, this paper integrates two-dimensional kernel principal component analysis (K2DPCA) plus two-dimensional linear discriminant analysis (2DLDA) (K2DPCA+2DLDA) into convolutional neural network (CNN) to recognize finger veins. Considering the row and column correlations of the finger vein image matrix and the classes of finger vein images, the authors adopted K2DPCA and 2DLDA separately for dimensionality reduction and extraction of nonlinear features in row and column directions, producing a dimensionally reduced compressed image without row or column correlation. Taking the dimensionally reduced compressed image as the input, the CNN was introduced to learn higher-level features, making finger vein recognition more accurate and robust. The public dataset of Finger Vein USM (FV-USM) Database was adopted for experimental verification. The results show that the proposed approach effectively overcome the common defects of original image feature extraction: the insufficient feature description, and the redundancy of information. When the training reached 120 epochs, the model basically realized stable convergence, with the loss approaching zero and the recognition rate reaching 97.3%. Compared with two-directional two-dimensional Fisher principal component analysis ((2D)2FPCA), our strategy, which integrates K2DPCA+2DLDA with CNN, achieved a very high recognition rate of finger vein images.

Robust two-dimensional capped [formula omitted]-norm linear discriminant analysis with regularization and its applications on image recognition

Two-dimensional linear discriminant analysis (2DLDA) is an effective matrix-based supervised dimensionality reduction method that expresses 2D data directly. However, 2DLDA magnifies the influence of outliers and noise since the construction of 2DLDA is based on squared Frobenius norm. To overcome its sensitivity, this paper investigates a two-dimensional capped l2,1-norm linear discriminant analysis, called 2DCLDA. The application of capped l2,1-norm makes 2DCLDA identify outliers and suppress the effect of noise effectively. To avoid singularity and give a stable performance, a regularization term is also considered in the between-class scatter. 2DCLDA is solved through a series of generalized eigenvalue problems, with each subproblem can be deemed as a weighted 2DLDA with regularization. It is proved that the proposed iteration algorithm monotonously decreases the objective of 2DCLDA. At last, 2DCLDA with its related approaches on several face image databases is compared, and the experimental results demonstrate the superiority of 2DCLDA, especially for noise data.

Generalized two-dimensional linear discriminant analysis with regularization

Recent advances show that two-dimensional linear discriminant analysis (2DLDA) is a successful matrix based dimensionality reduction method. However, 2DLDA may encounter the singularity issue theoretically, and also is sensitive to outliers. In this paper, a generalized Lp-norm 2DLDA framework with regularization for an arbitrary p>0 is proposed, named G2DLDA. There are mainly two contributions of G2DLDA: one is G2DLDA model uses an arbitrary Lp-norm to measure the between-class and within-class scatter, and hence a proper p can be selected to achieve robustness. The other one is that the introduced regularization term makes G2DLDA enjoy better generalization performance and avoid singularity. In addition, an effective learning algorithm is designed for G2LDA, which can be solved through a series of convex problems with closed-form solutions. Its convergence can be guaranteed theoretically when 1≤p≤2. Preliminary experimental results on three contaminated human face databases show the effectiveness of the proposed G2DLDA.

EEG Signal Classification Using Manifold Learning and Matrix‐Variate Gaussian Model

In brain-computer interface (BCI), feature extraction is the key to the accuracy of recognition. There is important local structural information in the EEG signals, which is effective for classification; and this locality of EEG features not only exists in the spatial channel position but also exists in the frequency domain. In order to retain sufficient spatial structure and frequency information, we use one-versus-rest filter bank common spatial patterns (OVR-FBCSP) to preprocess the data and extract preliminary features. On this basis, we conduct research and discussion on feature extraction methods. One-dimensional feature extraction methods like linear discriminant analysis (LDA) may destroy this kind of structural information. Traditional manifold learning methods or two-dimensional feature extraction methods cannot extract both types of information at the same time. We introduced the bilinear structure and matrix-variate Gaussian model into two-dimensional discriminant locality preserving projection (2DDLPP) algorithm and decompose EEG signals into spatial and spectral parts. Afterwards, the most discriminative features were selected through a weight calculation method. We tested the method on BCI competition data sets 2a, data sets IIIa, and data sets collected by our laboratory, and the results were expressed in terms of recognition accuracy. The cross-validation results were 75.69%, 70.46%, and 54.49%, respectively. The average recognition accuracy of new method is improved by 7.14%, 7.38%, 4.86%, and 3.8% compared to those of LDA, two-dimensional linear discriminant analysis (2DLDA), discriminant locality property projections (DLPP), and 2DDLPP, respectively. Therefore, we consider that the proposed method is effective for EEG classification.

Robust Two-Dimensional Linear Discriminant Analysis via Information Divergence

Due to the complexities of collected data that may contain outliers and noises, many variants of LDA and 2DLDA have been proposed to address this problem and have shown that the improved methods produce much better performance than classical LDA and 2DLDA. In this paper we propose a novel two-dimensional linear discriminant analysis method via information divergence. The proposed method applies the weighted L21 norm to learn a robust projection matrix in the image space. In the proposed model, we introduce the weights into the within-class scatter and the total scatter simultaneously, and learn the weights by imposing information divergence on the objective functions. To handle the proposed model, we resort to Dinkelbach’s extended algorithm to solve the proposed ratio minimization problem. Considering the characteristics of the subproblems, we exploit an equivalent representation of subproblems which can be solved by alternating optimization techniques where each block of variables has good optimization properties. The proposed model not only overcomes the small-sample-size problem, but also suppresses outliers by an adaptively weighted scheme with the guidance of information divergences. The experiments on several image data sets demonstrate that the classification performance of the proposed method is superior to that of some existing methods in the presence of outliers.

C2DNDA: A Deep Framework for Nonlinear Dimensionality Reduction

Dimensionality reduction has attracted much research interest in the past few decades. Existing dimensionality reduction methods like linear discriminant analysis and principal component analysis have achieved promising performance, but the single and linear projection properties limit further improvements of performance. A novel convolutional two-dimensional nonlinear discriminant analysis method is proposed for dimensionality reduction in this article. In order to handle nonlinear data properly, we present a newly designed structure with convolutional neural networks (CNNs) to realize an equivalent objective function with classical two-dimensional linear discriminant analysis (2DLDA) and thus embed the original 2DLDA into an end-to-end network. In this way, the proposed dimensionality reduction network can utilize the nonlinearity of the CNN and benefit from the learning ability. The results of experiment on different image-related applications demonstrate that our method outperforms other comparable approaches, and its effectiveness is proved.

Face Recognition Method Based on Probabilistic Neural Network Optimizing Two-Dimensional Subspace Analysis

If there is noise in the original image of face recognition, the efficiency of face recognition will be affected. In this paper, a face recognition method based on probabilistic neural network optimizing two-dimensional subspace analysis was proposed. Firstly, discrete wavelet variation was used to preprocess the image, and then two-dimensional linear discriminant analysis was used for feature extraction. Finally, the probabilistic neural network was used to complete the face classification. According to the results of experiments conducted on ORL and Fei general face database and the database collected independently, the recognition rate can also be as high as 98.9% when noise is added, and compared with several new identification methods, this method can achieve better identification performance.

Two-dimensional locality adaptive discriminant analysis

Two-dimensional Linear Discriminant Analysis (2DLDA), which is supervised and extracts the most discriminating features, has been widely used in face image representation and recognition. However, 2DLDA is inapplicable to many real-world situations because it assumes that the input data obeys the Gaussian distribution and emphasizes the global relationship of data merely. To handle this problem, we present a Two-dimensional Locality Adaptive Discriminant Analysis (2DLADA). Compared to 2DLDA, our method has two salient advantages: (1) it does not depend on any assumptions on the data distribution and is more suitable in real world applications; (2) it adaptively exploits the intrinsic local structure of data manifold. Performance on artificial dataset and real-world datasets demonstrate the superiority of our proposed method.

Robust bilateral Lp-norm two-dimensional linear discriminant analysis

Traditional linear discriminant analysis (LDA) may suffer from a sensitivity to outliers and the small sample size (SSS) problem, while the Lp-norm measure for 0 < p ≤ 1 is robust in a sense. In this paper, based on the criterion of the Bayes optimality, we propose a matrix-based bilateral Lp-norm two-dimensional linear discriminant analysis (BLp2DLDA) with robust performance, where 0 < p ≤ 1. We prove that the BLp2DLDA criterion is equivalent to an upper bound of the theoretical framework of the Bayes optimality. Compared with the L2-norm 2-directional 2-dimensional LDA ((2D)2LDA), our BLp2DLDA is more robust to outliers and noise. Moreover, unilateral Lp2DLDA can also be easily derived from BLp2DLDA. In addition, a simple but effective iterative technique is introduced to solve BLp2DLDA and the unilateral Lp2DLDA. Experimental results on different types of contaminated human face databases show that the proposed BLp2DLDA outperforms (2D)2LDA.

Rapid identification and quantification of cheaper vegetable oil adulteration in camellia oil by using excitation-emission matrix fluorescence spectroscopy combined with chemometrics

Camellia oil is a high quality oil mainly produced in southern China. It is common that unscrupulous merchants attempt to make huge profits by adulterating camellia oil with other cheaper or lower-quality vegetable oils. Therefore, this paper proposed excitation-emission matrix fluorescence spectroscopy combined with chemometric methods for the rapid identification and quantification of camellia oil adulteration with other cheaper vegetable oils. A five-component parallel factor analysis (PARAFAC) model roughly completed spectral characterization of oil samples, and obtained chemically meaningful information. Four advanced chemometrics methods were used for the classification of camellia oil and other vegetable oils (model 1) and the classification of camellia oil and adulterated camellia oil (models 2 and 3), respectively. Two-directional two-dimensional linear discriminant analysis ((2D)2LDA) was used for chemical data for the first time and showed huge potential. Furthermore, the developed N-PLS regression model used for the prediction of adulteration level in camellia oil showed satisfactory accuracy.

Fractional fuzzy 2DLDA approach for pomegranate fruit grade classification

In this research work, the fractional fuzzy two-dimensional linear discriminant analysis (FF2DLDA) an extension of fuzzy 2DLDA (F2DLDA) is proposed and applied to grade the two dimensional pomegranate fruit images. The research work investigates three existing and one novel mathematical feature extraction simulation technique to address the problem of non-destructive pomegranate fruit grading and classification. The approach is to use four feature extraction based mathematical simulation techniques, includes traditional 2DLDA, Fractional 2DLDA (FLDA), Fuzzy 2DLDA and FF2DLDA. The proposed technique holds the most discriminative features, by redefining the fuzzy between class scatter matrix of F2DLDA as fractional fuzzy between-class scatter matrix. In order to classify the extracted features, the kernel support vector machine (KSVM) along with all 2DLDA variants are used. The results show that FF2DLDA is manifold superior to the existing techniques, since the fractional fuzzy between class scatter matrix assigns a small fuzzy weight for edge classes and large fuzzy weight for non-edge classes. This effectively weakens the effect of edge class selection problem which is present in traditional 2D feature extraction techniques.

Nuclear-norm based 2DLDA with application to face recognition

Two-dimensional Linear Discriminant analysis (2DLDA) has been widely used for dimensionality reduction and feature extraction of data. However, most existing 2DLDA based methods cannot well characterize the spatial geometric structure of image. To solve this problem, inspired by the fact that nuclear-norm well reveals spatial relationship among pixels in each image and is robust to illumination and noise, we propose a nuclear-norm based two-dimensional Linear Discriminant analysis (2DLDA-nuclear), which employs nuclear-norm to characterize within-class and between-class scatters. Compared with most existing robust 2DLDA methods, our method not only helps suppress noise and illumination but also preserves spatial geometric structure of image. We also provide an effective iterative algorithm to solve 2DLDA-nuclear. Experiments on five public image databases show the effectiveness of our model.

A robust DCT-2DLDA watermark for color images

A blind watermarking algorithm is proposed, which is based on the Discrete Cosine Transform (DCT) method. It uses Two-Dimensional Linear Discriminant Analysis (2DLDA) watermark scheme for copyright protection. During the embedding process, the color image is converted into the YIQ color space. The quadrature chrominance component is transformed into frequency domain and it uses DCT method. Then, two binary watermarks for reference and logo are added to particular bits of the AC coefficients. During the extraction process, the logo watermark is extracted using matrix-based 2DLDA based on DCT method. By embedding the training data and using a matrix-based 2DLDA scheme, we do not need all the embedded information. The numbers of training samples for each class are small; the training covariance matrices are directly computed from 2D image samples by using 2DLDA. This ensures the efficient and the robustness of watermark extraction. Experimental results demonstrate that the differences between the watermarked image and the original image are indistinguishable. The proposed method is effectively resist common image processing attacks.

Algorithms for Face Image Mutual Reconstruction by Means of Two-Dimensional Projection Methods

In this paper, we consider the problem of mutual reconstruction of face image pairs. We addressed this problem in our previous article, where the proposed solutions were discussed in connection with Heterogeneous Face Recognition and Cross-Modal Multimedia Retrieval problems. Those solutions are based on one-dimensional and two-dimensional Principal Component Analysis performed over two original face images followed by their projection on independent eigenspaces, estimation of a transformation matrix and mutual reconstruction of the face image by means of one-dimensional and two-dimensional Karhunen-Loeve Transform. In this article, we propose new approaches and solutions, which are based solely on the two-dimensional eigenspace projection methods, and two regression models — Multiple Linear Regression and Partial Least Squares regression. We present the experiments on mutual reconstruction of face images in sketch/photo pairs, in pairs of face images with age-related changes, and in pairs of 2D/3D face images. In order to conduct the experiments, we selected two variants of the proposed approach. First one is based on two-dimensional Principal Component Analysis and Partial Least Squares regression, and the second one is based on two-dimensional Partial Least Squares and Multiple Linear Regression. Both variants showed acceptable performance for practical applications involving the mutual reconstruction of face images. Furthermore, we consider the method to improve the quality of reconstructed face images in the case of mixed datasets. This method involves classification of the dataset by means of two-dimensional Linear Discriminant Analysis and fitting of a separate regression model for each class. In addition, we show that generally, mutual reconstruction of face images is also achievable in conditions when original images are not a part of training sets of face images.