Principal Component Analysis Feature Selection Research Articles

Artificial intelligence-driven education evaluation and scoring: Comparative exploration of machine learning algorithms

Abstract With the widespread popularity of intelligent education, artificial intelligence plays an important role in the field of education. Currently, there are issues such as low accuracy and low adaptability. By comparing algorithms such as logistic regression, decision tree, random forest (RF), support vector machine, and long short-term memory (LSTM) recurrent neural network (RNN), this article adopted a multi-classification fusion strategy and fully considered the adaptability of the algorithm to evaluate and grade students in two scenarios with different grades and teaching quality. By encoding and normalizing student grades, six evaluation parameters were selected for the evaluation criteria of teaching quality through principal component analysis feature selection. Multi-classifier models were used to fuse the five models in pairs, improving the accuracy of the experimental evaluation. Finally, the experimental data of the six fused multi-classification models in the scenarios of student performance estimation and teaching quality estimation were compared, and the experimental effects of education evaluation and grading under different models were analyzed. The experimental results showed that the LSTM RNN-RF model had the strongest adaptability in the scenario of student performance estimation, with an estimation accuracy of 98.5%, which was 12.9% higher than a single RF model. This experiment was closely related to educational scenarios and fully considered the adaptability of different machine learning algorithms to different scenarios, improving the prediction and classification accuracy of the model.

PSMA PET/CT as a predictive tool for subregional importance estimates in the parotid gland

Objective. Xerostomia and radiation-induced salivary gland dysfunction remain a common side effect for head-and-neck radiotherapy patients, and attempts have been made to quantify the heterogeneity of the dose response within parotid glands. Prostate Specific Membrane Antigen (PSMA) ligands have demonstrated high uptake in salivary glands, which has been shown to correlate with gland functionality. Here we compare several models of parotid gland subregional relative importance with PSMA positron emission tomography (PET) uptake. We then develop a predictive model for Clark et al's relative importance estimates using PSMA PET and CT radiomic features, and demonstrate a methodology for predicting patient-specific importance deviations from the population. Approach. Intra-parotid gland uptake was compared with four regional importance models using 30 [18F]DCFPyL PSMA PET images. The correlation of uptake and importance was ascertained when numerous non-overlapping subregions were defined, while a paired t-test was used to compare binary region pairs. A radiomics-based predictive model of population importance was developed using a double cross-validation methodology. A model was then devised for supplementing population-level subregional importance estimates for each patient using patient-specific radiomic features. Main Results. Anticorrelative relationships were found to exist between PSMA PET uptake and four independent models of subregional parotid gland importance from the literature. Kernel Ridge Regression with principal component analysis feature selection performed best over test sets (Mean Absolute Error = 0.08), with gray level co-occurrence matrix (GLCM) features being particularly important. Deblurring PSMA PET images with neural blind deconvolution strengthened correlations and improved model performance. Significance. This study suggests that regions of relatively low PSMA PET uptake in parotid glands may exhibit relatively high dose-sensitivity. We’ve demonstrated the utility of PSMA PET radiomic features for predicting relative importance within subregions of parotid glands. PSMA PET appears to be a promising quantitative imaging modality for analyzing salivary gland functionality.

Comparison of structural analysis and principle component analysis for leakage prediction on superheater in boiler

<span>Leakage is one of the failures which commonly happens in boiler operation. Moreover, a continuous unsettled anomaly in a boiler could lead to leakage failure. An algorithm has been developed to predict the failure, consisting of three general procedures: feature selection, followed by hierarchical clustering, and naïve Bayes classification. The hierarchical clustering changes unlabeled data into labeled data, and naïve Bayes classification calculates the probability to justify anomaly occurrence. Meanwhile, this research focused on the effect of the feature selection method on the result of leakage prediction. Two different feature selection methods, namely the structural analysis and the principal component analysis (PCA), were deployed separately and then compared. The result showed that leakage prediction using the structural analysis method gave 13 hours 40 minutes of prediction time, and the PCA method gave 25 hours of prediction time. However, the PCA feature selection method caused more false alarms than feature selection with structural analysis, which only triggered five false alarms a week before leakage. Moreover, the structural analysis offered better traceability than PCA to understand the leakage occurrence.</span>

Performance Investigation of Principal Component Analysis for Intrusion Detection System Using Different Support Vector Machine Kernels

The growing number of security threats has prompted the use of a variety of security techniques. The most common security tools for identifying and tracking intruders across diverse network domains are intrusion detection systems. Machine Learning classifiers have begun to be used in the detection of threats, thus increasing the intrusion detection systems’ performance. In this paper, the investigation model for an intrusion detection systems model based on the Principal Component Analysis feature selection technique and a different Support Vector Machine kernels classifier is present. The impact of various kernel functions used in Support Vector Machines, namely linear, polynomial, Gaussian radial basis function, and Sigmoid, is investigated. The performance of the investigation model is measured in terms of detection accuracy, True Positive, True Negative, Precision, Sensitivity, and F-measure to choose an appropriate kernel function for the Support Vector Machine. The investigation model was examined and evaluated using the KDD Cup’99 and UNSW-NB15 datasets. The obtained results prove that the Gaussian radial basis function kernel is superior to the linear, polynomial, and sigmoid kernels in both used datasets. Obtained accuracy, Sensitivity, and, F-measure of the Gaussian radial basis function kernel for KDD CUP’99 were 99.11%, 98.97%, and 99.03%. for UNSW-NB15 datasets were 93.94%, 93.23%, and 94.44%.

Comparison of principal component analysis and ANFIS to improve EEVE Laboratory energy use prediction performance

The energy use that is in <span lang="EN-US">excess of practicum students’ needs and the disturbed comfort that the practicum students experience when conducting practicums in the Electrical eengineering vocational education (EEVE) laboratory. The main objective in this study was to figure out how to predict and streamline the use of electrical energy in the EEVE laboratory. The model used to achieve this research’s goal was called the adaptive neuro-fuzzy inference system (ANFIS) model, which was coupled with principal component analysis (PCA) feature selection. The use of PCA in data grouping performance aims to improve the performance of the ANFIS model when predicting energy needs in accordance with the standards set by the campus while still taking students’ confidence in conducting practicum activities during campus operating hours into consideration. After some experiments and tests, very good results were obtained in the training: R=1 in training; minimum RMSE=0.011900; epoch of 100 per iteration; and R=0.37522. In conclusion, the ANFIS model coupled with PCA feature selection was excellent at predicting energy needs in the laboratory while the comfort of the students during practicums in the room remained within consideration.</span>

A Novel Feature Selection Technique to Better Predict Climate Change Stage of Change

Indications of people’s environmental concern are linked to transport decisions and can provide great support for policymaking on climate change. This study aims to better predict individual climate change stage of change (CC-SoC) based on different features of transport-related behavior, General Ecological Behavior, New Environmental Paradigm, and socio-demographic characteristics. Together these sources result in over 100 possible features that indicate someone’s level of environmental concern. Such a large number of features may create several analytical problems, such as overfitting, accuracy reduction, and high computational costs. To this end, a new feature selection technique, named the Coyote Optimization Algorithm-Quadratic Discriminant Analysis (COA-QDA), is first proposed to find the optimal features to predict CC-SoC with the highest accuracy. Different conventional feature selection methods (Lasso, Elastic Net, Random Forest Feature Selection, Extra Trees, and Principal Component Analysis Feature Selection) are employed to compare with the COA-QDA. Afterward, eight classification techniques are applied to solve the prediction problem. Finally, a sensitivity analysis is performed to determine the most important features affecting the prediction of CC-SoC. The results indicate that COA-QDA outperforms conventional feature selection methods by increasing average testing data accuracy from 0.7% to 5.6%. Logistic Regression surpasses other classifiers with the highest prediction accuracy.

Machine Learning and Feature Selection Methods for EGFR Mutation Status Prediction in Lung Cancer

The evolution of personalized medicine has changed the therapeutic strategy from classical chemotherapy and radiotherapy to a genetic modification targeted therapy, and although biopsy is the traditional method to genetically characterize lung cancer tumor, it is an invasive and painful procedure for the patient. Nodule image features extracted from computed tomography (CT) scans have been used to create machine learning models that predict gene mutation status in a noninvasive, fast, and easy-to-use manner. However, recent studies have shown that radiomic features extracted from an extended region of interest (ROI) beyond the tumor, might be more relevant to predict the mutation status in lung cancer, and consequently may be used to significantly decrease the mortality rate of patients battling this condition. In this work, we investigated the relation between image phenotypes and the mutation status of Epidermal Growth Factor Receptor (EGFR), the most frequently mutated gene in lung cancer with several approved targeted-therapies, using radiomic features extracted from the lung containing the nodule. A variety of linear, nonlinear, and ensemble predictive classification models, along with several feature selection methods, were used to classify the binary outcome of wild-type or mutant EGFR mutation status. The results show that a comprehensive approach using a ROI that included the lung with nodule can capture relevant information and successfully predict the EGFR mutation status with increased performance compared to local nodule analyses. Linear Support Vector Machine, Elastic Net, and Logistic Regression, combined with the Principal Component Analysis feature selection method implemented with 70% of variance in the feature set, were the best-performing classifiers, reaching Area Under the Curve (AUC) values ranging from 0.725 to 0.737. This approach that exploits a holistic analysis indicates that information from more extensive regions of the lung containing the nodule allows a more complete lung cancer characterization and should be considered in future radiogenomic studies.

Towards feature selection for digital mammogram classification

The most common cancer type amongst women is the breast cancer with a large number of cases reported each year, many of them diagnosed in an advanced phase. In this paper our scope is to create a base for a support system that helps detecting breast cancer in an early stage. After defining the region of interest (ROI) and segmenting the result image (using k-means algorithm), Gray-Level Run-Length Matrices (GLRLM) features are extracted from both the ROI and from the segmented image in four directions (horizontal, vertical, first- and second diagonals). To reduce the dimensionality of the input data composed from the GLRLM features of the ROI and its segmentation for different combination of directions (removing redundant information, selecting just the most essential ones) two methods are used: Principal Component Analysis (PCA), and genetic algorithm (GA) feature selection. For classification, two methods are used and compared, namely Decision Trees (DT) and Random Forest (RF). For experiments we used the Mammographic Image Analysis Society (MIAS) dataset to train and to test the classifiers. The best performance is obtained for GLRLM features calculated for directions 45◦, and 90◦, using PCA feature selection and RF with a 100% training accuracy and 70% test accuracy.

Software Defect Prediction Based on Feature Subset Selection and Ensemble Classification

Two primary issues have emerged in the machine learning and data mining community: how to deal with imbalanced data and how to choose appropriate features. These are of particular concern in the software engineering domain, and more specifically the field of software defect prediction. This research highlights a procedure which includes a feature selection technique to single out relevant attributes, and an ensemble technique to handle the class-imbalance issue. In order to determine the advantages of feature selection and ensemble methods we look at two potential scenarios: (1) Ensemble models constructed from the original datasets, without feature selection; (2) Ensemble models constructed from the reduced datasets after feature selection has been applied. Four feature selection techniques are employed: Principal Component Analysis (PCA), Pearson’s correlation, Greedy Stepwise Forward selection, and Information Gain (IG). The aim of this research is to assess the effectiveness of feature selection techniques using ensemble techniques. Five datasets, obtained from the PROMISE software depository, are analyzed; tentative results indicate that ensemble methods can improve the model's performance without the use of feature selection techniques. PCA feature selection and bagging based on K-NN perform better than both bagging based on SVM and boosting based on K-NN and SVM, and feature selection techniques including Pearson’s correlation, Greedy stepwise, and IG weaken the ensemble models’ performance.

Bancos en Colombia: ¿Qué tan homogéneos son?

La homogeneidad, entendida como la falta de diversidad, es una fuente de fragilidad en sistemas complejos. Del mismo modo, la homogeneidad del sistema financiero ha sido documentada como un factor determinante del riesgo sistémico. En este documento se evalúa la homogeneidad en el caso colombiano, para lo cual se mide qué tan similares son los bancos según la estructura de sus estados financieros generales, así como de sus portafolios de cartera, de inversiones y de pasivos. La similitud entre bancos y una metodología de agrupamiento por aglomeración arrojan la estructura jerárquica del sistema bancario, la cual muestra cómo los bancos se relacionan entre ellos de acuerdo con su estructura financiera. El sector bancario colombiano muestra homogeneidad, en especial entre los bancos de mayor tamaño. Así mismo, es evidente que el tamaño es un factor importante en la estructura jerárquica de este sector. Los resultados son robustos a partir de un procedimiento de selección de variables basado en análisis de componentes principales, el cual reduce la dimensionalidad y redundancia de la base de datos. Los resultados permiten estudiar qué tan homogéneo es el sistema bancario, así como identificar aquellas instituciones bancarias que tienen una estructura financiera común (particular) y, por lo tanto, permiten estudiar de mejor manera el riesgo sistémico.

Intrusion Detection System using One Class SVM with and without Feature Selection in Wormhole Attack Detection

An Ad-hoc network is a kind of wireless construction from one to another computer, without having Wi-Fi access point or Router. However, the Ad hoc approach offers marginal security and decreases the data transfer rate. Consequently, it helps the attacker to connect with the ad-hoc network without any trouble. Therefore, a robust and reliable intrusion detection system (IDS) is a necessity of today’s information security domain. These IDS systems play a vital role in monitoring the threats encountered in a network by detecting the change in the normal profile due to attacks. Recently, to detect attacks the IDS are being equipped with machine learning algorithms to attain better accuracy and fast detection speed. Most of the IDS use different network features. However, enormous number of features makes the detection and prevention complicated. The IDS presented in this paper employs random forest and principal component analysis to minimize the number of features for network IDS for wireless ad hoc networks. The one class SVM has been used for detection of worm hole attack with and without feature selection. The performances of these approaches are compared with various existing techniques with false positive rate (FPR), accuracy and detection rate. Here, the accuracy improves and false positive rate reduces when intrusion is detected with feature selection technique. This paper discusses the performance of the one class SVM classifier in the wireless adhoc network IDS with random forest feature selection and principal component analysis feature selection techniques and one class SVM classifier without feature selection technique in the detection of wormhole attack. And the performance of one class SVM IDS is better in the detection of wormhole attack while it is implemented with principal component analysis feature selection technique.

PF-FELM: A Robust PCA Feature Selection for Fuzzy Extreme Learning Machine

Principal component analysis (PCA) is one of the crucial dimensionality reduction (DR) techniques in which the original features are transformed into lower dimensional space. Though the PCA space has orthogonal principal components (PC), it does not provide a real reduction of dimensionality in terms of the original features (variables), as all features including irrelevant and redundant features are required to define a single PC. It is necessary to remove such type of features by using feature subset selection (FSS) for better generalization performance. The principal intent of this paper is to introduce a PCA-based optimal FSS for fuzzy extreme learning machine (PF-FELM) approach that is able to handle weighted classification problem. The PF-FELM is mainly categorized into four subsystems: PCA, FSS, fuzzification, and classification. FSS selects an optimal feature order that is based on maximum occurrences within the various filter-based ranking algorithms. The simulation results are computed and compared by using sequential forward search strategy for clinical datasets. As a case study, an uncertain real-time application of plant disease and nutrient deficiency classification is considered, helping remote farmers as expert advice. For validation of the proposed PF-FELM, statistical and hypothetical tests are evaluated by using correlation measure and paired t-test. With the results, it is inferred that PF-FELM provides 12.17% improved generalization performance as compared to P-FELM (without FSS) by reducing 13.94% features.

Optimal BP neural network algorithm for state of charge estimation of lithium-ion battery using PSO with PCA feature selection

The state of charge (SOC) is the residual capacity of a battery, which indicates the available charge left inside a battery to drive a vehicle. Accurate SOC estimation is of great significance for a lithium-ion battery to ensure its safe operation and to prevent it from over-charging or over-discharging. However, it is difficult to get an accurate value of SOC since it is an inner state of a battery cell, which cannot be directly measured. This paper presents an improved SOC estimation strategy for a lithium-ion battery using the back-propagation neural network (BPNN). Two algorithms, principal component analysis (PCA) and particle swarm optimization (PSO), are used to enhance the accuracy and robustness. PCA is utilized to select the most significant input features. The PSO algorithm is developed to determine the optimal value of hidden layer neurons and the learning rate since these parameters are the most critical factors in constructing an optimal BPNN model. The proposed model is tested and evaluated by using three electric vehicle drive cycles. The performance of the proposed model is compared with common BPNN and radial basis function neural network (RBFNN) models and verified based on the root mean square error (RMSE), mean square error (MSE), mean absolute error (MAE), mean absolute percentage error (MAPE), and SOC error. The validation results are very effective in predicting SOC with very narrow SOC error which demonstrates the model robustness. The results indicate that the proposed model computes RMSE to be 0.58%, 0.72%, and 0.47% for the Beijing Dynamic Stress Test (BJDST), Federal Urban Drive Schedule (FUDS), and US06, cycle, respectively.

A Machine Learning Approach to Detect Router Advertisement Flooding Attacks in Next-Generation IPv6 Networks

Router advertisement (RA) flooding attack aims to exhaust all node resources, such as CPU and memory, attached to routers on the same link. A biologically inspired machine learning-based approach is proposed in this study to detect RA flooding attacks. The proposed technique exploits information gain ratio (IGR) and principal component analysis (PCA) for feature selection and a support vector machine (SVM)-based predictor model, which can also detect input traffic anomaly. A real benchmark dataset obtained from National Advanced IPv6 Center of Excellence laboratory is used to evaluate the proposed technique. The evaluation process is conducted with two experiments. The first experiment investigates the effect of IGR and PCA feature selection methods to identify the most contributed features for the SVM training model. The second experiment evaluates the capability of SVM to detect RA flooding attacks. The results show that the proposed technique demonstrates excellent detection accuracy and is thus an effective choice for detecting RA flooding attacks. The main contribution of this study is identification of a set of new features that are related to RA flooding attack by utilizing IGR and PCA algorithms. The proposed technique in this paper can effectively detect the presence of RA flooding attack in IPv6 network.

Tooth Color Detection Using PCA and KNN Classifier Algorithm Based on Color Moment

Matching the suitable color for tooth reconstruction is an important step that can make difficulties for the dentists due to the subjective factors of color selection. Accurate color matching system is mainly result based on images analyzing and processing techniques of recognition system.  This system consist of three parts, which are data collection from digital teeth color images, data preparation for taking color analysis technique and extracting the features, and data classification involve feature selection for reducing the features number of this system. The teeth images which is used in this research are 16 types of teeth that are taken from RSGM UNAIR SURABAYA. Feature extraction is taken by the characteristics of the RGB, HSV and LAB based on the color moment calculation such as mean, standard deviation, skewness, and kurtosis parameter. Due to many formed features from each color space, it is required addition method for reducing the number of features by choosing the essential information like Principal Component Analysis (PCA) method. Combining the PCA feature selection technique to the clasification process using K Nearest Neighbour (KNN) classifier algorithm can be improved the accuracy performance of this system. On the experiment result, it showed that only using KNN classifier achieve accuracy percentage up to 97.5 % in learning process and 92.5 % in testing process while combining PCA with KNN classifier can reduce the 36 features to the 26 features which can improve the accuracy percentage up to 98.54 % in learning process and  93.12% in testing process. Adding PCA as the feature selection method can be improved the accuracy performance of this color matching system with little number of features.Â

PCA Feature Selection For Recognition of Chatter Gestation

A SVM(Support Vector Machine)-like framework provides a novel way to learn linear principal component analysis (PCA), which is easy to be solved and can obtain the unique global solution. SVM is good at classification and PCA features is introduced into SVM. So, a new recognition method based on hybrid PCA and SVM is proposed and used for a series of experiments on chatter gestation. The results of chatter gestation recognition and chatter prediction experiments are presented and show that the method proposed is effective.