Bi-2DPCA: A Fast Face Coding Method for Recognition

Overview of principal component analysis algorithm

Recently, support vector machine (SVM) has become a popular tool in time series forecasting. In developing a successful SVM forecaster, feature extraction is the first important step. This paper proposes the applications of principal component analysis (PCA), kernel principal component analysis (KPCA) and independent component analysis (ICA) to SVM for feature extraction. PCA linearly transforms the original inputs into uncorrelated features. KPCA is a nonlinear PCA developed by using the kernel method. In ICA, the original inputs are linearly transformed into statistically independent features. By examining the sunspot data and one real futures contract, the experiment shows that SVM by feature extraction using PCA, KPCA or ICA can perform better than that without feature extraction. Furthermore, there is better generalization performance in KPCA and ICA feature extraction than PCA feature extraction.

A comparison of PCA, KPCA and ICA for dimensionality reduction in support vector machine

Reducing the complexity of high-dimensional acoustic data is essential for effective vehicle recognition, especially in intelligent transportation systems. This study evaluated six dimension reduction techniques, including Principal Component Analysis (PCA), Kernel PCA, Incremental PCA, Independent Component Analysis (ICA), Truncated Singular Value Decomposition (SVD), and Latent Dirichlet Allocation (LDA), to address the challenges of data redundancy while maintaining relevant features. The dataset includes acoustic signals from seven categories of vehicles extracted using Mel-Frequency Cepstral Coefficients (MFCC), Spectral Centroid, and Spectral Bandwidth. Incremental PCA showed the highest accuracy (0.982) on scenarios with larger training datasets, with effective management of high-dimensional data. ICAs provide optimal performance with fewer components at a higher proportion of test data, demonstrating their efficiency in retaining information. SVD shows stability across all data ratios, confirming its reliability for a wide range of applications. Although LDAs maintain competitive results, their interpretability stands out in certain tasks. These findings emphasize the importance of selecting appropriate dimension reduction methods based on data characteristics and application needs, providing valuable insights to improve the accuracy and efficiency of vehicle recognition systems. 

Remote sensing datasets and methods are suitable for mapping and managing the natural resources like minerals, clean water, and energy and also govern their sustainability nowadays. Hyperspectral (HS) imaging has immense potential for rock type classification, mineral mapping, and identification. This work demonstrates the potential of feature extraction techniques and unsupervised machine learning methods for the space-borne hyperspectral remote sensing data in characterizing and identifying mineral and classifying rock type in Banswara, Rajasthan, India. Feature extraction techniques can reveal variations within the data, which can help identify geological areas, reduce noise, and check the dimensionality of the data. Singular value decomposition (SVD)-based principal component analysis (PCA), kernel PCA (KPCA), minimum noise fraction (MNF), and independent component analysis (ICA) were tested for lithological mapping using recently launched DLR Earth Sensing Imaging Spectrometer Hyperspectral (DESIS) and PRecursore IperSpettrale della Missione Applicativa (PRISMA) data in order to map geologically significant areas. Unsupervised machine learning methods, such as Iterative Self-Organizing Data Analysis Technique (ISODATA) and K-means, were also employed. Vertex component analysis (VCA) was utilized to check for similarity and identify various spectral features. Our work demonstrates the advantages of using feature extraction algorithms such as PCA and KPCA over MNF and ICA in geological mapping and interpretability. We recommend K-means as the preferred method for lithological classification of hyperspectral remote sensing data. Our work highlights the potential of advanced feature extraction algorithms for mineral mapping using hyperspectral data, providing different ways to identify minerals and ultimately leading to better mineral resource management.

Feature extraction methods such as Kernel Principal Component Analysis (KPCA) and Principal Component Analysis (PCA), are often used for soft sensor modeling in industrial process with high dimensional data. A kind of soft sensor method based on Integrated Principal Component Analysis (Integrated PCA) is proposed for some weakness of KPCA and that of PCA. This approach combines nonlinear information extracted by KPCA with linear information extracted by PCA and it can not only reduce the dimensionality of input data, but also make full use of linear and nonlinear information. Partial Least Squares (PLS) is used to obtain the final soft sensor model and Particle Swarm Optimization (PSO) is applied to get the optimal parameters of Integrated PCA and those of KPCA. Finally, the proposed method is applied to build soft sensor models of diesel oil boiling point and other industrial objects and is proved to have better ability of generalization by being compared with other feature extraction methods.

The anomaly detection is applicable to wide range of critical infrastructure elements due to frequent change in anomaly occurrences and make sure to avoid all threats identified in regular. In this perception, we have to identify the abnormal patterns in applications and to model them by using a new adorned machine learning classifiers. In this paper we are investigating the performance by comparison of heterogeneous machine learning classifiers: ICA (Independent Component Analysis), LDA (Linear Discriminant Analysis), PCA (Principal Component Analysis), Kernel PCA and other learning classifiers. The Kernel PCA (KPCA) is a non-linear extension to PCA used to classify the data and detect anomalies by orthogonal transformation of input space into (usually high dimensional) feature space. The KPCA use kernel trick by extract the principal components from set of corresponding eigenvectors and use kernel width as performance parameter to determine rate of classification. The KPCA is implemented on taking two UCI machine learning repository sets and one real bank dataset. The KPCA implemented with classic Gaussian kernel internally. Finally KPCA performance compared with projection methods (ICA, LDA, PLSDA and PCA), other kernel (SVM-K) and non-kernel techniques (ID3, C4.5, Rule C4.5, k-NN and NB) applied on same datasets using training and test set combinations.

Face recognition is a state-of-the-art widely employed in areas where security is a concern. The technology consists of two steps namely feature extraction and face classification. In this paper, different dimensionality reduction algorithms such as Convolutional Autoencoder (CAE), Principal Component Analysis (PCA), Polynomial Kernel-Principal Component Analysis (KPCA) and Independent Component Analysis (ICA) were explored and compared on the LFW dataset. After the face features were extracted by the mentioned techniques, a Support Vector Machine (SVM) was used to perform face recognition by classifying the images into 7 groups. It was found out that the accuracy of the SVM was highly dependent on the feature extraction technique employed, with the CAE proving to be the most superior algorithm leading to a classification accuracy of 81.9%. The ICA was the next best performing technique followed by the KPCA and finally by the PCA.

In this study, we delve into the dynamics of the Sri Lankan government bond market, building upon prior research that focused on the application of principal component analysis (PCA) in modelling sovereign yield curves. Our analysis encompasses data spanning from January 2010 to August 2022. The study applied several PCA variants such as multivariate PCA, Randomized PCA, Incremental PCA, Sparse PCA, Functional PCA, and Kernel PCA on smoothed data. Kernel PCA was found to explain the majority of the variation associated with the data. Findings reveal that the first principal component accounted for a substantial 97.69% of the variations in yield curve movements, 2nd PCA accounted for 1.88%, and 3rd for 0.42%. These results align with previous research, which generally posits that the initial three principal components tend to elucidate around 95% of the fluctuations within the term structure of yields. Our results question the empirical findings, which state that the 1st PCA represents the longer tenor of the yield curve. In Sri Lanka, instead, the 1st PCA represents the 3-year bond yields. It may be because of the liquidity constraints in underdeveloped frontier markets, where longer tenor yields do not react fast enough to reflect the movement of the yield curve. The 2nd PCA represents the slope of the yield curve which is the yield difference of a 10-year T-Bond and 3 months T-Bill. The 3rd PCA which represents the curvature of the yield curve attributed to 2 × 3 years T-Bond yield—3 months T-bill10-year T-Bond.

Feature Extraction (EF) is considered the effective process among all the data processing steps of the classification system. In real-life applications, the reliability of a classifier is highly affected by high-dimensional irrelevant and redundant information. Hence extraction of appropriate data plays an imperative role to reduce the dimensionality and increase the performance of the classification system. Herein paper, a hybrid Principal Independent Component Analysis (PICA) technique is presented by the combination of the two most popular Principal Component Analysis (PCA) and Singular Value Decomposition (SVD) feature extraction techniques. The authors execute the proposed PICA technique with the SGD classifier of machine learning (ML) and analyze the performance by comparing the results with existing PCA, LDA, SVD, and ICA feature extraction techniques. Furthermore, to evaluate the PICA's performance, results are compared without applying any feature extraction techniques or with existing ICA, PCA, LDA, and SVD methods. The effectiveness of the presented work is better than existing work found in the literature and is considered on an improved scale of accomplished 3.94% accuracy, 1.35% Sensitivity, 7.70% Specificity, and 5.27% precision. Moreover, decrease the 42.60% RMSE and 15% dimensionality.

Nowadays, behind wall human detection based on UWB radar signal, which it had a strong anti-jamming performance, was an important problem. In this setting, principal component analysis (PCA) as an anomaly detection method was used, but PCA could only deal with linear data. Thus, we introduced the kernel principal component analysis (KPCA) for performing a nonlinear form of principal component analysis (PCA). We obtained the different state data based on UWB radar signal for the behind wall human detection. These data were used as training and testing data to calculate the squared prediction error (SPE) values that detect anomalies. The experimental results showed that the introduced approach of KPCA effectively captured the nonlinear relationship in the process data and showed superior process monitoring performance compared to linear PCA.

Face recognition efficiency is extremely challenged by image corruption, noise, shadowing and variant face expressions. In this paper, we propose a reliable incremental face recognition algorithm to address this problem. The algorithm is robust to face image misalignment, occlusion, corruption and different style variations. To apply this for large face data streams, the proposed algorithm uses incremental robust principal component analysis (PCA) to regain the intrinsic data of a bunch of images regarding one subject. A novel similarity metric is defined for face recognition and classification. Five different databases and a base of four different criteria are used in the experiments to illustrate the reliability of the proposed method. Experiments point that it outperforms other existing incremental PCA approaches namely incremental singular value decomposition, add block singular value decomposition and candid covariance-free incremental PCA in terms of recognition rate under occlusions, facial expressions and image perspectives.

ASD (Autism Spectrum Disorder) is a neurological disorder that causes lifelong disturbances in children resulting in mental illness. Treatment can help but cannot be cured. Currently ASD is detected by understanding a child's behavior and intellectual activity. This diagnosis can be subjective, time-consuming, inconclusive, does not provide precise insight into genetics and is unsuitable for early detection. In Autism, a major challenge faced in many healthcare conditions is timing of diagnosis. It can take up to 6 months to diagnose a child with autism with certainty because of the lengthy process, and a child must see many different specialists to diagnose autism, from a developmental pediatrician, neurologist, psychiatrist or psychologist. Machine Learning Methods can make relevant changes to speed up the process. In this study, it is proposed to apply PCA (Principal Component Analysis). PCA is basically the basis of multivariate data analysis that applies the projection method. This analysis technique is usually used to summarize multivariate data tables on a large scale so that they can be used as a collection of smaller variables or a summary index. From there, variables are then analyzed to find out certain trends, variable clusters, and outliers. In this study it is proposed to implement the PCA (Principal Component Analysis) algorithm, namely PCA (Principal Component Analysis), Kernel PCA, Sparse PCA, and Incremental PCA. In this study using the experimental method by making applications to implement the proposed algorithm. Then test the model using the secondary dataset and measure the performance of the model. The research results show that the model that applies Sparse PCA gives the best results, which means that the application of PCA can be used to reduce the number of features and increase model performance.

The classic principal components analysis (PCA), kernel PCA (KPCA) and linear discriminant analysis (LDA) feature extraction methods evaluate the importance of components according to their covariance contribution, not considering the entropy contribution, which is important supplementary information for the covariance. To further improve the covariance-based methods such as PCA (or KPCA), this paper firstly proposed an entropy matrix to load the uncertainty information of random variables similar to the covariance matrix loading the variation information in PCA. Then an entropy-difference matrix was used as a weighting matrix for transforming the original training images. This entropy-difference weighting (EW) matrix not only made good use of the local information of the training samples, contrast to the global method of PCA, but also considered the category information similar to LDA idea. Then the EW method was integrated with PCA (or KPCA), to form new feature extracting method. The new method was used for face recognition with the nearest neighbor classifier. The experimental results based on the ORL and Yale databases showed that the proposed method with proper threshold parameters reached higher recognition rates than the usual PCA (or KPCA) methods.

This study presents an effective data-driven anomaly detection scheme for drunk driving detection. Specifically, the proposed anomaly detection approach amalgamates the desirable features of the t-distributed stochastic neighbor embedding (t-SNE) as a feature extractor with the Isolation Forest (iF) scheme to detect drivers’ drunkenness status. We used the t-SNE model to exploit its capacity in reducing the dimensionality of nonlinear data by preserving the local and global structures of the input data in the feature space to obtain good detection. At the same time, the iF scheme is an effective and unsupervised tree-based approach to achieving good detection of anomalies in multivariate data. This approach only employs normal events data to train the detection model, making them more attractive for detecting drunk drivers in practice. To verify the detection capacity of the proposed t-SNE-iF approach in reliably detecting drivers with excess alcohol, we used publically available data collected using a gas sensor, temperature sensor, and a digital camera. The overall detection system proved a high detection performance with AUC around 95%, demonstrating the proposed approach’s robustness and reliability. Furthermore, compared to the Principal Component Analysis (PCA), Incremental PCA (IPCA), Independent component analysis (ICA), Kernel PCA (kPCA), and Multi-dimensional scaling (MDS)-based iForest, EE, and LOF detection schemes, the proposed t-SNE-based iF scheme offers superior detection performance of drunk driver status.