Principal Component Analysis Dimensionality Reduction Research Articles

Principal Component Analysis

Principal component analysis (PCA) is often applied for analyzing data in the most diverse areas. This work reports, in an accessible and integrated manner, several theoretical and practical aspects of PCA. The basic principles underlying PCA, data standardization, possible visualizations of the PCA results, and outlier detection are subsequently addressed. Next, the potential of using PCA for dimensionality reduction is illustrated on several real-world datasets. Finally, we summarize PCA-related approaches and other dimensionality reduction techniques. All in all, the objective of this work is to assist researchers from the most diverse areas in using and interpreting PCA.

Matching Subsequence Music Retrieval in a Software Integration Environment

This paper firstly introduces the basic knowledge of music, proposes the detailed design of a music retrieval system based on the knowledge of music, and analyzes the feature extraction algorithm and matching algorithm by using the features of music. Feature extraction of audio data is the important research of this paper. In this paper, the main melody features, MFCC features, GFCC features, and rhythm features, are extracted from audio data and a feature fusion algorithm is proposed to achieve the fusion of GFCC features and rhythm features to form new features under the processing of principal component analysis (PCA) dimensionality reduction. After learning the main melody features, MFCC features, GFCC features, and rhythm features, based on the property that PCA dimensionality reduction can effectively reduce noise and improve retrieval efficiency, this paper proposes vector fusion by dimensionality reduction of GFCC features and rhythm features. The matching retrieval of audio features is an important task in music retrieval. In this paper, the DTW algorithm is chosen as the main algorithm for retrieving music. The classification retrieval of music is also achieved by the K-nearest neighbor algorithm. In this paper, after implementing the research and improvement of algorithms, these algorithms are integrated into the system to achieve audio preprocessing, feature extraction, feature postprocessing, and matching retrieval. This article uses 100 different kinds of MP3 format music as the music library and randomly selects 4 pieces each time, and it tests the system under different system parameters, recording duration, and environmental noise. Through the research of this paper, the efficiency of music retrieval is improved and theoretical support is provided for the design of music retrieval software integration system.

Evaluation of pattern recognition techniques for the attribution of cultural heritage objects based on the qualitative XRF data

Various instrumental and experimental modifications of portable XRF (pXRF) spectrometers meet nearly all the requirements for safe and efficient analysis of objects of cultural heritage, allowing this technique to be the most used analytical method for that purpose. The technique was used in a joint campaign (involving scientists, curators and art historians) to examine and protect an iconostasis containing well-preserved and reliably dated icons from four different centuries. The number of acquired XRF spectra was sufficient to be treated with pattern recognition techniques. Different pattern recognition techniques were applied in the analysis of cultural heritage object, both unsupervised and supervised. The results of various analytical examinations (non-invasive or performed on samples) pretreated or not, even a fusion of different techniques, were subjected to pattern recognition analysis.The possibilities of select pattern recognition techniques to analyze data obtained by a pXRF spectrometer from the surfaces of paintings and icons are presented in this study. Principal Component Analysis (PCA), Hierarchical Cluster Analysis (HCA), and Scattering Matrix Based Dimension Reduction (SMBDR) were used to process generated datasets of peak areas and spectral intensities providing dimension reduction of the initial datasets. The efficiency of this process is evaluated and discussed using the calculated index of informativeness and Bhattacharyya’s distance. Based on the results, the possibility of classification is discussed in terms of choosing proper classifiers, which were designed thereafter. The classification is evaluated by recognition ability and prediction ability parameters, and then tested by the leave-one-out procedure and using pXRF spectra collected from different color and support (cardboard paintings). An efficient procedure was developed for the attribution of artwork origin, based solely on XRF spectral data and PCA dimension reduction, followed by linear classification.

Comparative Analysis of Machine Learning Techniques with Principal Component Analysis on Kidney and Heart Disease

Detection of disease at earlier stages is the most challenging one. Datasets of different diseases are available online with different number of features corresponding to a particular disease. Many dimensionality reduction and feature extraction techniques are used nowadays to reduce the number of features in dataset and finding the most appropriate ones. This paper explores the difference in performance of different machine learning models using Principal Component Analysis dimensionality reduction technique on the datasets of Chronic kidney disease and Cardiovascular disease. Further, the authors apply Logistic Regression, K Nearest Neighbour, Naïve Bayes, Support Vector Machine and Random Forest Model on the datasets and compare the performance of the model with and without PCA. A key challenge in the field of data mining and machine learning is building accurate and computationally efficient classifiers for medical applications. With an accuracy of 100% in chronic kidney disease and 85% for heart disease, KNN classifier and logistic regression were revealed to be the most optimal method of predictions for kidney and heart disease respectively.

Info_PCA: A Hybrid Technique to Improve Accuracy by Dimensionality Reduction

Principal Component Analysis and Shannon Entropy are some of the most widely used methods for feature extraction and selection. PCA reduces the data to a new subspace with low dimensions by calculating the eigenvectors from eigenvalues out of a covariance matrix and thereby reduces the features to a smaller number capturing the significant information. Shannon entropy is based on probability distribution to calculate the significant information content. Information gain shows the importance of a given attribute in the set of feature vectors. The paper has introduced a hybrid technique Info_PCA which captures the properties of Information gain and PCA that overall reduces the dimensionality and thereby increases the accuracy of the machine learning technique. It also demonstrates the individual implementation of Information gain for feature selection and PCA for dimensionality reduction on two different datasets collected from the UCI machine learning repository. One of the major aims is to determine the important attributes in a given set of training feature vectors to differentiate the classes. The paper has shown a comparative analysis on the classification accuracy obtained by the application of Information Gain, PCA and Info_PCA applied individually on the two different datasets for feature extraction followed by ANN classifier where the results of hybrid technique Info_PCA achieves maximum accuracy and minimum loss in comparison to other feature extraction techniques.

Bearing fault diagnosis method based on similarity measure and ensemble learning

Ensemble learning aggregates the predictions of diverse predictors. Most existing ensemble learning methods gather the base classifiers based on the weight of accuracy. However, if the weight is adjusted only according to the correct rate of the base classifiers, it ignores the different types of errors caused by different working principles of the base classifiers. Blindly taking the correct rate of the basis classifier as the weight will increase the probability of the classifier making the same mistake and thus reduce the correct rate. To address this problem, this paper proposes a multi-round voting method based on similarity measurement. In the proposed method, the characteristic parameters of the data are extracted and PCA dimensionality reduction is used to train several base classifiers, which achieved high accuracy of about 90% at first. Then, we extract the confusion matrix of these base classifiers to facilitate the calculation of the similarity measurement matrix and the obtainment of the similarity matrix between base classifiers. Later, on the basis of the number of base classifiers and the values of the similarity matrix, we set a certain number of similar thresholds. Finally, according to the comparison of similarity value and similarity thresholds, a multi-round voting classifier is trained. By comparing the hard voting classifier, multi-round voting classifier and their base classifiers, the experimental results verify that the multi-round voting classifier has higher accuracy than other methods.

A Novel End-To-End Feature Selection and Diagnosis Method for Rotating Machinery.

Feature selection is to obtain effective features from data, also known as feature engineering. Traditional feature selection and predictive model learning are separated, and there is a problem of inconsistency of criteria. This paper presents an end-to-end feature selection and diagnosis method that organically unifies feature expression learning and machine prediction learning into one model. The algorithm first combines the prediction model to calculate the mean impact value (MIVs) of the feature and realizes primary feature selection for the prediction model by selecting the feature with a larger MIV. In order to take into account the performance of the feature itself, the within-class and between-class discriminant analysis (WBDA) method is proposed, and combined with the feature diversity strategy, the feature-oriented secondary selection is realized. Eventually, feature vectors obtained by two selections are classified using a multi-class support vector machine (SVM). Compared with the modified network variable selection algorithm (MIVs), the principal component analysis dimensionality reduction algorithm (PCA), variable selection based on compensative distance evaluation technology (CDET), and other algorithms, the proposed method MIVs-WBDA exhibits excellent classification accuracy owing to the fusion of feature selection and predictive model learning. According to the results of classification accuracy testing after dimensionality reduction on rotating machinery status, the MIVs-WBDA method has a 3% classification accuracy improvement under the low-dimensional feature set. The typical running time of this classification learning algorithm is less than 10 s, while using deep learning, its running time will be more than a few hours.

Hyperspectral Image Classification Based on GS-CatBoost Model

Aiming at the features of hyperspectral imges with large amount of redundant information of hyperspectral images, small correlation between features and categories, small sample size, and high-dimensions data, a model of hyperspectral image classification based on an improved mRMR band selection method and GS-CatBoost is proposed. First of all, using the mRMR improved by K-L to select the optimal band subset with high correlation between features and category, low redundancy, and large amount of information. Then, the Gabor filter is embedded in PCA and LDA combined dimensionality reduction to complete the extraction of space-spectrum joint features. Finally, using grid search to optimize the CatBoost algorithm to classify the extracted features. Experiments show that the proposed model fully and effectively extracts the space-spectrum joint features of hyperspectral images. In the Salinas Scene dataset, OA is 97.87%, AA is 99.01%, and Kappa coefficient is 0.9763.

Research and Application of Intersection Clustering Algorithm Based on PCA Feature Extraction and K-Means

Traffic jam and traffic route design are important issues of people’s livelihood. In this paper, machine learning algorithm is used to cluster intersection features. Based on the collected traffic intersection data, PCA dimension reduction technology is used for feature extraction, and then K-Means clustering algorithm is used to sum up the number of entrances, and each intersection is clustered. Thus, a new intersection clustering algorithm is established, studied and applied. The algorithm can match similar intersections, which provides theoretical support for the transportation department and related technical personnel.

Neural network in sports cluster analysis

In the era of rapid development of the Internet, various types of data in daily life are becoming more and more important, as are people's sports data. How to collect and store these massive exercise data and how to extract the user's exercise habits from these data are of great significance for improving people's exercise enthusiasm. This article mainly studies the application of the network in the cluster analysis of sports. This paper proposes a combination of a neural network-based sports data analysis model and a density peak clustering algorithm for unsupervised dimensionality reduction of high-dimensional data. We design both the encoder and the decoder with a three-layer fully connected neural network structure. The encoder extracts the characteristics of the sample data, and then, the decoder approximates the original input sample. The encoder is used to reduce the dimensionality of the high-dimensional data to the middle dimension, combined with the density peak clustering algorithm to further reduce the dimensionality, and then analyze the low-dimensional data. And set the corresponding learning rate for different data sets, the three data sets are iterated 40 times. The prediction accuracy rates of the algorithm in the three data sets are 94.1%, 90.3%, and 89.6% respectively; compared with the traditional PCA dimensionality reduction method, the method in this paper can extract data features more effectively, improve the dispersion between clusters, and have a better clustering effect. Finally, a numerical example is given to illustrate the effectiveness of the proposed algorithm.

A Detailed Analysis on Kidney and Heart Disease Prediction using Machine Learning

Detection of disease at earlier stages is the most challenging one. Datasets of different diseases are available online with different number of features corresponding to a particular disease. Many dimensionalities reduction and feature extraction techniques are used nowadays to reduce the number of features in dataset and finding the most appropriate ones. This paper explores the difference in performance of different machine learning models using Principal Component Analysis dimensionality reduction technique on the datasets of Chronic kidney disease and Cardiovascular disease. Further, the authors apply Logistic Regression, K Nearest Neighbour, Naïve Bayes, Support Vector Machine and Random Forest Model on the datasets and compare the performance of the model with and without PCA. A key challenge in the field of data mining and machine learning is building accurate and computationally efficient classifiers for medical applications. With an accuracy of 100% in chronic kidney disease and 85% for heart disease, KNN classifier and logistic regression were revealed to be the most optimal method of predictions for kidney and heart disease respectively.

Multi-Angle Movie Reviews Analysis Based on Multi Model

In this study, movie reviews are used as data sets to extract related phrases, topics, and sentiment scores from the text. Based on users’ information, users’ behavior preferences and their influences are analysed, and text semantic information is mined from multiple perspectives. A variety of data processing and machine learning methods including text segmentation, Apriori association rule mining algorithm, sentiment analysis, linear fitting, TFIDF algorithm, PCA dimensionality reduction, and LDA topic model is used in the research. At the same time, due to the coarse granularity of the topic extraction in the LDA algorithm, it is not suitable for short text, this paper proposes a new topic model based on improved k-means and TextRank and gets good results on this dataset. This paper uses multiple data mining models to analyse film reviews and presents an empirical study of the efficacy of machine learning techniques in text semantic mining.

Identification of Leaf-Scale Wheat Powdery Mildew (Blumeria graminis f. sp. Tritici) Combining Hyperspectral Imaging and an SVM Classifier

Powdery mildew (PM, Blumeria graminis f. sp. tritici) is a devastating disease for wheat growth and production. It is highly meaningful that the disease severities can be objectively and accurately identified by image visualization technology. In this study, an integral method was proposed based on a hyperspectral imaging dataset and machine learning algorithms. The disease severities of wheat leaves infected with PM were quantitatively identified based on hyperspectral images and image segmentation techniques. A technical procedure was proposed to perform the identification and evaluation of leaf-scale wheat PM, specifically including three primary steps of the acquisition and preprocessing of hyperspectral images, the selection of characteristic bands, and model construction. Firstly, three-dimensional reduction algorithms, namely principal component analysis (PCA), random forest (RF), and the successive projections algorithm (SPA), were comparatively used to select the bands that were most sensitive to PM. Then, three diagnosis models were constructed by a support vector machine (SVM), RF, and a probabilistic neural network (PNN). Finally, the best model was selected by comparing the overall accuracies. The results show that the SVM model constructed by PCA dimensionality reduction had the best result, and the classification accuracy reached 93.33% by a cross-validation method. There was an obvious improvement of the identification accuracy with the model, which achieved an 88.00% accuracy derived from the original hyperspectral images. This study can provide a reference for accurately estimating the disease severity of leaf-scale wheat PM and other plant diseases by non-contact measurement technology.

146-OR: Specific Metabolomic Signatures Associate with Diabetes Remission after Weight Loss

The response to weight loss intervention is heterogeneous both in weight loss amount and improvement in metabolism. We seek to identify biomarkers predictive of long-term weight loss and type 2 diabetes (T2D) remission in the Longitudinal Assessment of Bariatric Surgery (LABS) and the Look AHEAD (Action for Health in Diabetes) studies. Methods: LABS is a longitudinal observational study of 2458 consecutive cases (2006-2009) of bariatric surgery. Look AHEAD is an RCT comparing an intensive lifestyle intervention to T2D support and education in 5,145 U.S. adults with obesity and T2D enrolled between 2001-2004. Targeted mass spectrometry-based profiling of 135 metabolites was performed in pre-intervention blood samples from individuals selected based on sustained T2D remission (N=93 LABS, N=80 Look AHEAD) and extremes (top and bottom 13th percentile) of weight loss (N=151 LABS, 23 with T2D) at 4 or 5 years. We used PCA for dimensionality reduction of metabolites and logistic regression models to determine association between metabolic factors and phenotype. Improvement in model fit and net reclassification index analyses were performed to determine if metabolites improved a clinical model for predicting remission. Results: One factor composed of branched chain amino acids (BCAA) and tyrosine (p=0.045), and one composed of betaine and choline (p=0.02) were associated with T2D remission. Inclusion of these factors in a clinical prediction model improved the fit (p=0.04). Only one factor, composed of C31:1-C37:1 sphingomyelins, associated with weight loss at 5 years (p=0.036). Conclusions. Our findings define novel associations between metabolites in the BCAA and trimethylamine-N-oxide-microbiome-related pathways that are independently and incrementally associated with sustained T2D remission after weight loss. Future studies of these analytes could help in the discovery of metabolic regulatory mechanisms, and guide more personalized weight loss interventions in individuals with T2D. Disclosure L.C. Kwee: None. O. Ilkayeva: None. M. Muehlbauer: None. B.M. Wolfe: None. J.Q. Purnell: Advisory Panel; Self; Novo Nordisk A/S. H. Chen: None. C.B. Newgard: None. S.H. Shah: Research Support; Self; AstraZeneca, Verily Life Sciences LLC. Funding National Institute of Diabetes and Digestive and Kidney Diseases (R01DK108580)

Hyperspectral Estimation of Soil Organic Matter Content using Different Spectral Preprocessing Techniques and PLSR Method

Soil organic matter (SOM) is the main source of soil nutrients, which are essential for the growth and development of agricultural crops. Hyperspectral remote sensing is one of the most efficient ways of estimating the SOM content. Visible, near infrared, and mid-infrared reflectance spectroscopy, combined with the partial least squares regression (PLSR) method is considered to be an effective way of determining soil properties. In this study, we used 54 different spectral pretreatments to preprocess soil spectral data. These spectral pretreatments were composed of three denoising methods, six data transformations, and three dimensionality reduction methods. The three denoising methods included no denoising (ND), Savitzky–Golay denoising (SGD), and wavelet packet denoising (WPD). The six data transformations included original spectral data, R; reciprocal, 1/R; logarithmic, log(R); reciprocal logarithmic, log(1/R); first derivative, R’; and first derivative of reciprocal, (1/R)’. The three dimensionality reduction methods included no dimensionality reduction (NDR), sensitive waveband dimensionality reduction (SWDR), and principal component analysis (PCA) dimensionality reduction (PCADR). The processed spectra were then employed to construct PLSR models for predicting the SOM content. The main results were as follows—(1) the wavelet packet denoising (WPD)-R’ and WPD-(1/R)’ data showed stronger correlations with the SOM content. Furthermore, these methods could effectively limit the correlation between the adjacent bands and, thus, prevent “overfitting”. (2) Of the 54 pretreatments investigated, WPD-(1/R)’-PCADR yielded the model with the highest accuracy and stability. (3) For the same denoising method and spectral transformation data, the accuracy of the SOM content estimation model based on SWDR was higher than that of the model based on NDR. Furthermore, the accuracy in the case of PCADR was higher than that for SWDR. (4) Dimensionality reduction was effective in preventing data overfitting. (5) The quality of the spectral data could be improved and the accuracy of the SOM content estimation model could be enhanced effectively, by using some appropriate preprocessing methods (one combining WPD and PCADR in this study).

Near-field sound source localization using principal component analysis–multi-output support vector regression

In this article, principal component analysis method, which is applied to image compression and feature extraction, is introduced into the dimension reduction of input characteristic variable of support vector regression, and a method of joint estimation of near-field angle and range based on principal component analysis dimension reduction is proposed. Signal-to-noise ratio and calculation amount are the decisive factors affecting the performance of the algorithm. Principal component analysis is used to fuse the main characteristics of training data and discard redundant information, the signal-to-noise ratio is improved, and the calculation amount is reduced accordingly. Similarly, support vector regression is used to model the signal, and the upper triangular elements of the signal covariance matrix are usually used as input features. Since the covariance matrix has more upper triangular elements, training it as a feature input will affect the training speed to some extent. Principal component analysis is used to reduce the dimensionality of the upper triangular element of the covariance matrix of the known signal, and it is used as the input feature of the multi-output support vector regression machine to construct the near-field parameter estimation model, and the parameter estimation of unknown signal is herein obtained. Simulation results show that this method has high estimation accuracy and training speed, and has strong adaptability at low signal-to-noise ratio, and the performance is better than that of the back-propagation neural network algorithm and the two-step multiple signal classification algorithm.

Detection and recognition of human body posture in motion based on sensor technology

The recognition of human motion posture is of great values in the field of sports. In this paper, inertial sensor technology is employed to recognize four postures including dribbling, passing, catching, and shooting in basketball. The data are collected by four nine‐axis inertial sensors worn on the arm. The time domain and frequency domain features are extracted after the process of smoothing and normalizing. Then 30 eigenvectors are obtained by dimensionality reduction of principal component analysis (PCA), and the support vector machine (SVM) method is used for posture recognition. The experimental results show that after PCA dimensionality reduction, the recognition accuracy of the features as the SVM input is significantly high. Compared with the recognition accuracy of back propagation neural network (BPNN) (85.4%), the average recognition accuracy of SVM is 96%, which verifies the reliability of the method proposed in this study. This research proves the effectiveness of sensor technology in basketball posture recognition, which can provide a reliable guidance for basketball training. © 2020 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Handwritten Digital Image Classification Based on PCA Dimensionality Reduction

Dimensionality reduction is an important idea in machine learning. In the machine learning, some high-dimensional data sets are often encountered. In the case of high-dimensional data, data samples are sparse and distance calculations are difficult. Such problems are serious problems faced by all machine learning methods. “Dimensional disaster.” In addition, linear correlation between features is easy to occur in high-dimensional features, which means that some features are redundant.

Deep Fusion Feature Extraction and Classification of Pellet Phase

Pellet quality including chemical composition, physical properties and metallurgical performance of three parts, its quality and mineral composition, properties and structure of the pellets has the close relation, studies show that the mineralogical micro structure and distribution of pellets had significant effects on the metallurgical properties, so the analysis and determination of pellets of mineral composition and micro structure is very important to improve the quality of pellets. Paper to pellets micro mineral as the research object, mainly studies the CNN and PCA two kinds of image processing algorithm, in the heart of the traditional model of CNN structure characteristics obtained by convolution PCA dimension reduction at a time, will be the main features of PCA to extract into the depth of the CNN learning, realize the mineral phase of shallow and deep information of the image to do effective fusion, in a larger extent, reflects the mineralogical characteristics, thus more intuitive response metallurgical properties. The location and alkalinity of the ore phase were identified by the extracted deep fusion feature, and the results were compared with those of the traditional CNN algorithm. It was found that the accuracy of location and alkalinity recognition of PCA and CNN coupling algorithm was 93.82% and 91.26%, respectively, which were higher than the accuracy of traditional CNN algorithm 92.73% and 88.93%, which verified the accuracy of PCA and CNN coupling model and its applicability in mineral phase recognition.

The Research on Face Recognition and Segmentation Based on Intelligent Background

Affected by factors such as attitude, light, expression, etc., it is impossible to accurately identify the identity in a wireless visual sensor network in an uncontrollable environment. In traditional visual identity recognition, it is necessary to convert uncontrollable factors into controllable and stable feature factors for identity recognition in a relatively uncontrollable environment where the node distribution is relatively complicated. The conversion process leads to long recognition time and low efficiency. An adaptive recognition method for identity features in wireless visual sensing networks based on LBP face recognition is proposed. A strong classifier is obtained for cascading, and the underlying features are extracted. The final Harr face cascade classifier is applied to the face Check it out. The PCA dimensionality reduction processing of the facial area feature vector is performed to obtain the low-dimensional feature vector, the dimensionality reduction coefficient, and the average face of the person. For the face image in the wireless local area, its LBP operation is given. Perform histogram statistics on face feature information, obtain face LBP histograms, and perform feature matching on the face feature database to complete recognition. The improved algorithm has improved the cumulative matching score of traditional algorithms by 17.8%; the accuracy rate has improved It is 32.7%, and the recognition time is shortened by 3.9s. Simulation results show that the proposed algorithm has high accuracy and recognition efficiency.