Correlation-based Feature Selection Method Research Articles

ANCONE: An Interactive System for Mining and Visualization of Students’ Information in the Context of PLANEA 2015

Data mining has been widely used in different areas of knowledge, and education is no exception. Data mining uses computer models to analyze data and answer research questions to help in decision making. This article uses data from the PLANEA 2015 Mathematics in Middle school (last year of Middle school) test, which measures the academic achievement and provides a personal, family and school context, in order to find those characteristics that are related to the academic level of the tested students. In this article, an interactive visualization system was developed that allows observing interesting patterns and association rules by combining relevant attributes (variables) and the States. To reduce the analysis space, the Correlation-Based Feature Selection method was used to reduce categorical and numerical attributes. The results show a significant reduction (93%) in the number of attributes, with very little loss of information, when certain attributes are eliminated. Particularly, the 232 categorical attributes obtained from each student are reduced to only 18 attributes, which are correlated with the results of students in the PLANEA test. In addition, empirically it was discovered that choosing the mode from the labels of plausible values as the target class increases the accuracy in classifiers used to show the goodness of the reduction obtained. Some of the relevant attributes are the "AcademicAspiration", "FamilyResources", "MotherStudies" and "FatherStudies". From the 30 States with information, only 8 are in the Basic level, the other States are in the Below Basic level .

ANCONE: An Interactive System for Mining and Visualization of Students’ Information in the Context of PLANEA 2015

Data mining has been widely used in different areas of knowledge, and education is no exception. Data mining uses computer models to analyze data and answer research questions to help in decision making. This article uses data from the PLANEA 2015 Mathematics in Middle school (last year of Middle school) test, which measures the academic achievement and provides a personal, family and school context, in order to find those characteristics that are related to the academic level of the tested students. In this article, an interactive visualization system was developed that allows observing interesting patterns and association rules by combining relevant attributes (variables) and the States. To reduce the analysis space, the Correlation-Based Feature Selection method was used to reduce categorical and numerical attributes. The results show a significant reduction (93%) in the number of attributes, with very little loss of information, when certain attributes are eliminated. Particularly, the 232 categorical attributes obtained from each student are reduced to only 18 attributes, which are correlated with the results of students in the PLANEA test. In addition, empirically it was discovered that choosing the mode from the labels of plausible values as the target class increases the accuracy in classifiers used to show the goodness of the reduction obtained. Some of the relevant attributes are the AcademicAspiration, FamilyResources, MotherStudies and FatherStudies. From the 30 States with information, only 8 are in the Basic level, the other States are in the Below Basic level .

GIS Based Hybrid Computational Approaches for Flash Flood Susceptibility Assessment

Flash floods are one of the most devastating natural hazards; they occur within a catchment (region) where the response time of the drainage basin is short. Identification of probable flash flood locations and development of accurate flash flood susceptibility maps are important for proper flash flood management of a region. With this objective, we proposed and compared several novel hybrid computational approaches of machine learning methods for flash flood susceptibility mapping, namely AdaBoostM1 based Credal Decision Tree (ABM-CDT); Bagging based Credal Decision Tree (Bag-CDT); Dagging based Credal Decision Tree (Dag-CDT); MultiBoostAB based Credal Decision Tree (MBAB-CDT), and single Credal Decision Tree (CDT). These models were applied at a catchment of Markazi state in Iran. About 320 past flash flood events and nine flash flood influencing factors, namely distance from rivers, aspect, elevation, slope, rainfall, distance from faults, soil, land use, and lithology were considered and analyzed for the development of flash flood susceptibility maps. Correlation based feature selection method was used to validate and select the important factors for modeling of flash floods. Based on this feature selection analysis, only eight factors (distance from rivers, aspect, elevation, slope, rainfall, soil, land use, and lithology) were selected for the modeling, where distance to rivers is the most important factor for modeling of flash flood in this area. Performance of the models was validated and compared by using several robust metrics such as statistical measures and Area Under the Receiver Operating Characteristic (AUC) curve. The results of this study suggested that ABM-CDT (AUC = 0.957) has the best predictive capability in terms of accuracy, followed by Dag-CDT (AUC = 0.947), MBAB-CDT (AUC = 0.933), Bag-CDT (AUC = 0.932), and CDT (0.900), respectively. The proposed methods presented in this study would help in the development of accurate flash flood susceptible maps of watershed areas not only in Iran but also other parts of the world.

Client evaluation decision models in the credit scoring tasks

Abstract One of the important decision-making problems of modern financial institutions is credit scoring, which involves assessing credit risk. Decision-making models based on classifiers and feature selection methods that reduce the complexity of a decision problem by limiting the number of conditional attributes find use in such problems. The article examines the effectiveness of various combinations of classifiers and feature selection methods in the problem of credit risk assessment. The results of the conducted research indicate that for the considered set of data on cash loans granted, the Correlation-based Feature Selection method is the best method among the considered ones, and the Random Forest is the most effective classifier.

Automatic Classification of Microseismic Records in Underground Mining: A Deep Learning Approach

The identification of suspicious microseismic events is the first crucial step in processing microseismic data. In this paper, we present an automatic classification method based on a deep learning approach for classifying microseismic records in underground mines. A total of 35 commonly used features in the time and frequency domains were extracted from waveforms. To examine the discriminative ability of these features, a genetic algorithm (GA)-optimized correlation-based feature selection (CFS) method was applied. As a result, 11 features were selected to represent microseismic records. By dividing each microseismic record into 50 frames, an 11 × 50 feature matrix was utilized as the input. A convolutional neural network (CNN) with 35 layers was trained on 20,000 samples recorded at the Huangtupo Copper and Zinc Mine. There are 5 types of events: microseismic events, blasting, ore extraction, mechanical noise, and electromagnetic interference. The event type was correctly determined by the trained CNN classifier 98.2% of the time, outperforming traditional machine learning methods.

Classification and Prediction of Tibetan Medical Syndrome Based on the Improved BP Neural Network

Tibetan medicine has a long history as a traditional ethnic medicine in China. It is playing an important role in the medical system in northwestern of China, and which has attracted more and more attention due to its unique diagnostic system and clinical efficacy. Meanwhile, as the data mining technology has been widely used in traditional Chinese medicine (TCM), its application in the field of Tibetan medicine has also launched preliminarily. In this paper, we are focusing on Chronic Atrophic Gastritis (CAG) which is a typical gastrointestinal disease in the plateau area, and a novel back-propagation (BP) network model is proposed for Tibetan medical syndrome classification and prediction. K-means clustering algorithm was firstly implemented on the diagnostic data which was obtained from the Qinghai Provincial Tibetan Hospital, and then Correlation-based Feature Selection (CFS) method was adopted for feature selection. The selected feature vectors were finally put into the proposed BP network for training and testing. In order to overcome BP network's typical shortcomings including slow convergence and easy to overfit, we use a method based on Gaussian distribution to improve weights initialization, and dynamically adjusted the learning rate using the learning rate exponential decay method. Further, we add regularization to the loss function to prevent overfitting. Ultimately, the experiment achieved an accuracy of 99.09%, which improved significantly after improvement and achieved better result compared with other classification methods.

Feature selection for human membrane protein type classification using filter methods

<span lang="EN-US">As the number of protein sequences in the database is increasing, effective and efficient techniques are needed to make these data meaningful. These protein sequences contain redundant and irrelevant features that cause lower classification accuracy and increase the running time of the computational algorithm. In this paper, we select the best features using Minimum Redundancy Maximum Relevance(mRMR) and Correlation-based feature selection(CFS) methods. Two datasets of human membrane protein are used, S1 and S2. After the features have been selected by mRMR and CFS, K-Nearest Neighbor(KNN) and Support Vector Machine(SVM) classifiers are used to classify these membrane proteins. The performance of these techniques is measured using accuracy, specificity and sensitivity. and F-measure. The proposed algorithm managed to achieve 76% accuracy for S1 and 73% accuracy for S2. Finally, our proposed methods present competitive results when compared with the previous works on membrane protein classification</span><span>.</span>

High-Performance Feature Selection Model for Network Intrusion Detection System

Network intrusions detection is a continuous vigilant task and to efficiently analyze the traffic in the corporate network to detect network intrusions. The efficiency of the Network Intrusion Detection System (NIDS) performance can be improved by adopting feature selection or reduction process to suit the present day high speed real time networks. This work is focused on identifying the key features of the audit dataset used to build an efficient light-weight NIDS. The NSL KDD dataset is used in this work titled Attribute Richness Based Feature Selection (ARFS) in order to analyze its performance.The obtained results are compared with the Correlation-based Feature Selection (CFS) and Information Gain (IG) feature selection methods. The proposed feature selection method produced better detection rate comparatively.

Sensor-Assisted Weighted Average Ensemble Model for Detecting Major Depressive Disorder.

The present methods of diagnosing depression are entirely dependent on self-report ratings or clinical interviews. Those traditional methods are subjective, where the individual may or may not be answering genuinely to questions. In this paper, the data has been collected using self-report ratings and also using electronic smartwatches. This study aims to develop a weighted average ensemble machine learning model to predict major depressive disorder (MDD) with superior accuracy. The data has been pre-processed and the essential features have been selected using a correlation-based feature selection method. With the selected features, machine learning approaches such as Logistic Regression, Random Forest, and the proposed Weighted Average Ensemble Model are applied. Further, for assessing the performance of the proposed model, the Area under the Receiver Optimization Characteristic Curves has been used. The results demonstrate that the proposed Weighted Average Ensemble model performs with better accuracy than the Logistic Regression and the Random Forest approaches.

Classification of Sambas Traditional Fabric “Kain Lunggi” Using Texture Feature

Traditional fabric is a cultural heritage that has to be preserved. Kain Lunggi is Sambas traditional fabric that saw a decline in its crafter. To introduce Kain Lunggi in a broader national and global society in order to preserve it, a digital image processing based system to perform Kain Lunggi pattern recognition need to be built. Feature extraction is an important part of digital image processing. The visual feature that does not represent the character of an object will affect the accuracy of a recognition system. The purposes of this research are to perform feature selection on sets of feature to determine the best feature that can increase recognition accuracy. This research conducted in several steps which are image acquisition of Kain Lunggi pattern, preprocessing to reduce image noise, feature extraction to obtain image features, and feature selection. GLCM is implemented as a feature extraction method. Feature extraction result will be used in a feature selection process using CFS (Correlation-based Feature Selection) methods. Selected features from CFS process are Angular Second Moment, Contrast, and Correlation. Selected features evaluation is conducted by calculating classification accuracy with the KNN method. Classification accuracy prior to feature extraction is 85.18% with K values K=1 ; meanwhile, the accuracy increases to 88.89% after feature selection. The highest accuracy improvement of 20.74% in KNN occurred when using K value K= 4.

Railway turnout system RUL prediction based on feature fusion and genetic programming

The remaining useful life (RUL) prediction of railway turnout systems (RTS) is very important to avoid unplanned shutdowns and reduce labor costs for the normal operation of railways. One key challenge on RUL prediction is how to construct an appropriate health indicator (HI) that can be utilized to infer conditions of RTS. Existing methods usually adopt some inherit merits (e.g., monotonicity, trendability, and robustness), and their prediction results lack real-world physical meaning due to their “black-box-like” property. In this paper, we present a novel feature fusion method for RUL prediction, which is able to capture the relationship between RUL and HI. A variant correlation-based feature selection method is utilized to extract features, which has the potential to depict the degradation process optimally, and then the selected features are fused by Auto-Associative Kernel Regression (AAKR) for prediction. To reduce the noise interference, the extracted features and the combined HI are all smoothed by using the locally weighted regression. Finally, a genetic programming (GP) algorithm is employed to predict the RUL of RTS. The proposed method is extensively tested on two turnout machine degradation datasets, and the results show that the proposed approach is effective for RUL prediction of RTS.

Multi-sensor fusion based on multiple classifier systems for human activity identification

Multimodal sensors in healthcare applications have been increasingly researched because it facilitates automatic and comprehensive monitoring of human behaviors, high-intensity sports management, energy expenditure estimation, and postural detection. Recent studies have shown the importance of multi-sensor fusion to achieve robustness, high-performance generalization, provide diversity and tackle challenging issue that maybe difficult with single sensor values. The aim of this study is to propose an innovative multi-sensor fusion framework to improve human activity detection performances and reduce misrecognition rate. The study proposes a multi-view ensemble algorithm to integrate predicted values of different motion sensors. To this end, computationally efficient classification algorithms such as decision tree, logistic regression and k-Nearest Neighbors were used to implement diverse, flexible and dynamic human activity detection systems. To provide compact feature vector representation, we studied hybrid bio-inspired evolutionary search algorithm and correlation-based feature selection method and evaluate their impact on extracted feature vectors from individual sensor modality. Furthermore, we utilized Synthetic Over-sampling minority Techniques (SMOTE) algorithm to reduce the impact of class imbalance and improve performance results. With the above methods, this paper provides unified framework to resolve major challenges in human activity identification. The performance results obtained using two publicly available datasets showed significant improvement over baseline methods in the detection of specific activity details and reduced error rate. The performance results of our evaluation showed 3% to 24% improvement in accuracy, recall, precision, F-measure and detection ability (AUC) compared to single sensors and feature-level fusion. The benefit of the proposed multi-sensor fusion is the ability to utilize distinct feature characteristics of individual sensor and multiple classifier systems to improve recognition accuracy. In addition, the study suggests a promising potential of hybrid feature selection approach, diversity-based multiple classifier systems to improve mobile and wearable sensor-based human activity detection and health monitoring system.

Analyzing the Bandgap of 3D Perovskite Oxides: Machine Learning Approach

In this study we have developed a machine learning approach to analyze and predict the bandgaps of 1146 ABO3 perovskites, computed using density functional theory. Since DFT-computed bandgaps with high accuracy are normally also highly resource demanding, we developed a machine learning model to predict the perovskite bandgaps by utilizing readily available atomic and crystal descriptors. These descriptors include atomic electrochemical properties such as electron affinity and ionization energy of the A/B-site atoms, as well as Goldschmidt tolerance factor of the crystal structure. As we arrive at our results, we conduct a comparative study of two machine learning algorithms: namely kernel ridge regression and gradient boosting regression. Along the way, we employ recursive feature elimination and correlation based feature selection methods to remove the weakest inputs from our dataset, and reduce the 35 starting features down to 28 most essential features, resulting in a final model mean square error of 0.2 eV. Lastly, we performed a relative contribution analysis of the inputs with regards to their influence on the bandgap of the perovskites. In essence, we attempt to use machine learning as an analytical technique for evaluating the structure-bandgap relationship of perovskite.

Distinguishing Different Emotions Evoked by Music via Electroencephalographic Signals.

Music can evoke a variety of emotions, which may be manifested by distinct signals on the electroencephalogram (EEG). Many previous studies have examined the associations between specific aspects of music, including the subjective emotions aroused, and EEG signal features. However, no study has comprehensively examined music-related EEG features and selected those with the strongest potential for discriminating emotions. So, this paper conducted a series of experiments to identify the most influential EEG features induced by music evoking different emotions (calm, joy, sad, and angry). We extracted 27-dimensional features from each of 12 electrode positions then used correlation-based feature selection method to identify the feature set most strongly related to the original features but with lowest redundancy. Several classifiers, including Support Vector Machine (SVM), C4.5, LDA, and BPNN, were then used to test the recognition accuracy of the original and selected feature sets. Finally, results are analyzed in detail and the relationships between selected feature set and human emotions are shown clearly. Through the classification results of 10 random examinations, it could be concluded that the selected feature sets of Pz are more effective than other features when using as the key feature set to classify human emotion statues.

Defeating Untrustworthy Testing Parties: A Novel Hybrid Clustering Ensemble Based Golden Models-Free Hardware Trojan Detection Method

Due to the globalization of the design and fabrication process of integrated circuits (ICs), ICs are becoming vulnerable to hardware Trojans. Most of the existing hardware Trojan detection works assume that the testing stage is trustworthy. However, testing parties may collude with malicious attackers and modify the results of hardware Trojan detection. In this paper, two attack models for untrustworthy testing parties are formulated. We further propose an adversarial data generation method for untrustworthy testing parties to modify the collected test data. Then, we propose a novel hybrid clustering ensemble method to build a trusted hardware Trojan detection method (clustering ensemble-based hardware Trojan detection method) against untrustworthy testing parties. To alleviate the impact of process variations and noises on hardware Trojan detection in the actual measurement, the unsupervised correlation-based feature selection method is exploited to process the raw test data of ICs for feature selection. The proposed method can eliminate the need of the fabricated golden chips and the simulated golden models. It can also resist the malicious modifications on Trojan detection results introduced by untrustworthy testing parties. Besides, the following problems and questions are also theoretically analyzed and answered: 1) the number of necessary testing parties; 2) the time overhead and the computational overhead of the proposed method; 3) how to choose the basic clustering algorithms (by using a proposed diversity analysis algorithm); and 4) the reason why the proposed clustering ensemble method is superior to the majority voting method. Both the EDA evaluation on ISCAS89 benchmarks and field-programmable gate array evaluation on Trust-HUB benchmarks are performed to evaluate the performance of the proposed method. Experimental results demonstrate that the proposed method can resist malicious modifications robustly and can detect hardware Trojans with high accuracy (up to 93.75%). Meanwhile, the introduced time overhead is small.

A study on visual features of leaves in plant identification using artificial intelligence techniques

In this paper, artificial intelligence techniques (AIT) such as artificial neural network, naive bayes algorithm, random forest algorithm, K-nearest neighborhood (KNN) and support vector machine (SVM) are implemented to design an automatic identifier for the plant leaves. For this purpose, data of 637 healthy leaves consisting of 32 different plant species are used. 22 visual features (VF) of each leaf are extracted by using image processing techniques. These 22 VF are considered in 4 groups including dimension (D#6), color (C#6), texture (T#5) and pattern (P#5). In order to investigate the effects of these groups on the classifying performance, 15 possible different combinations from the 4 groups are constituted. The models are then trained via the data of 510 leaves, and their accuracy are tested through the data of 127 leaves. From the results of the test, SVM model with the accuracy of 92.91% is found to be the most successful identifier for combination including all groups. The next best result is achieved with the accuracy of 87.40% for the combination of D#6, C#6 and P#5 groups. Since the most important issue in the classification process is the use of the minimum number of VF, 16 most effective VF on the identification are determined by means of correlation-based feature selection (CFS) method. The best result for these 16 VF is also achieved with the accuracy of 94.49% by the SVM model. Then the performance of the proposed method is tested to identify the diseased and defected leaves. Therefore, 637 healthy and 33 diseased/defected leaves are put together. Randomly selected 536 leaves corresponding to 80% of all leaves are used for training and the remaining 134 leaves are used for testing, and identified with the accuracy of 92.53% by the SVM model. With this study, it is numerically revealed that the P#5 is the most effective feature group. Moreover, it has been determined that the most effective feature in the P#5 group is the feature of edge Fourier transform. The results point out that, if AIT models are properly modelled and trained, they can be successfully and effectively applied to the identification of the plants even if there are diseased and defected samples.

Effective Promotional Strategies Selection in Social Media: A Data-Driven Approach

Nowdays, many companies, organizations and individuals are using the function of sharing or retweeting information to promote their products, policies, and ideas on social media. While a growing body of research has focused on identifying the promoters from millions of users, the promoters themselves are seeking to know which strategy can improve promotional effectiveness, which is rarely studied in the literature. In this work, we investigate an open problem of effective promotional strategy selection via causal analysis which is challenging in identifying and quantifying promotional strategies as well as the selection bias when estimating the causal effect of promotional strategies from observational data. We study the promotional strategies not only on the content level (what to promote) but also on the context level (when and how to promote). To alleviate the issue of selection bias in observational studies, we propose a data-driven approach that is a Propensity Score Matching (PSM) based method, which helps to evaluate the causal effect of each promotional strategy and discover the set of effective strategies to predict the promotional effectiveness (i.e., the number of users infected by the promotion). We evaluate our proposed method on a real social dataset including 194 million users and 5 million promoted messages. Experimental results show that (1) the top-ranked strategies by our PSM based method significantly and consistently outperform the correlation based feature selection methods in predicting promotional effectiveness; (2) we conclude our observations from the real data with three interpretable and practical ideas for steering social media promotion.

Commercial Video Evaluation via Low-Level Feature Extraction and Selection

To discover the influence of the commercial videos’ low-level features on the popularity of the videos, the feature selection method should be used to get the video features influencing the videos’ evaluation mostly after analyzing the source data and the audiences’ evaluations of the videos. After extracting the low-level features of the videos, this paper improved the Correlation-Based Feature Selection (CFS) method which is widely used and proposed an algorithm named CFS-Spearmen which combined the Spearmen correlation coefficient and the classical CFS to select features. The 4 datasets in UCI machine learning database were employed as the experiment data. The experiment results were compared with the results using traditional CFS, Minimum Redundancy and Maximum Relevance (mRMR). The SVM was used to test the method in this paper. Finally, the proposed method was used in commercial videos’ feature selection and the most influential feature set was obtained.

Integrated network analysis and machine learning approach for the identification of key genes of triple-negative breast cancer.

Triple-negative breast cancer (TNBC) has attracted more attention compared with other breast cancer subtypes due to its aggressive nature, poor prognosis, and chemotherapy remains the mainstay of treatment with no other approved targeted therapy. Therefore, the study aimed to discover more promising therapeutic targets and investigating new insights of biological mechanism of TNBC. Six microarray data sets consisting of 463 non-TNBC and 405 TNBC samples were mined from Gene Expression Omnibus. The data sets were integrated by meta-analysis and identified 1075 differentially expressed genes. Protein-protein interaction network was constructed which consists of 486 nodes and 1932 edges, where 29 hub genes were obtained with high topological measures. Further, 16 features (hub genes), 12 upregulated (AURKB, CCNB2, CDC20, DDX18, EGFR, ENO1, MYC, NUP88, PLK1, PML, POLR2F, and SKP2) and four downregulated ( CCND1, GLI3, SKP1, and TGFB3) were selected through machine learning correlation based feature selection method on training data set. A naïve Bayes based classifier built using the expression profiles of 16 features (hub genes) accurately and reliably classify TNBC from non-TNBC samples in the validation test data set with a receiver operating curve of 0.93 to 0.98. Subsequently, Gene Ontology analysis revealed that the hub genes were enriched in mitotic cell cycle processes and Kyoto Encyclopedia of Genes and Genomes pathway analysis showed that they were enriched in cell cycle pathways. Thus, the identified key hub genes and pathways highlighted in the study would enhance the understanding of molecular mechanism of TNBC which may serve as potential therapeutic target.

State of health estimation of second-life LiFePO4 batteries for energy storage applications

This paper focuses on the identification of the aging mechanism and estimation of the state of health of second-life batteries. Six retired LiFePO4 batteries are selected to conduct cycle life tests under three typical load profiles for energy storage applications. By adopting incremental capacity analysis (ICA) and IC peak area analysis, aging mechanisms in the batteries are studied. All the batteries have shown the same aging pattern with a combination of loss of lithium inventory (LLI) and loss of active materials on negative electrodes (LAMNE). The LLI and LAMNE are analyzed in a quantitative manner to detect the similarities and differences among the batteries operated under different load profiles. To estimate the battery remaining capacity, three types of regression methods are proposed and compared. The features of IC curves are used as inputs to the regression models. The results show that the estimation errors with ordinary least squares (OLS) regression and ridge regression methods are within 2%, and that ridge regression has lower root mean square error than OLS regression. Using correlation-based feature selection methods, a universal index that is feasible for all batteries is presented for regression analysis, and the estimation error is found to be within 3%.