Correlation-based Feature Selection Method Research Articles

Improving phase prediction accuracy for high entropy alloys with Machine learning

High entropy alloys (HEAs) can possess improved physicochemical properties due to the formation of solid solution (SS), intermetallic (IM) and/or mixed (SS + MM) phases with unique microstructures. For this reason, phase prediction in HEAs (SS, IM, or SS + IM) is the preliminary step in alloy design. In this research, main-stream data vetting techniques and Machine Learning (ML) algorithms have been used to classify and predict the phases in HEAs with the goal of significantly improving phase prediction accuracy. The data used for this work collected from literature are first refined manually by removing redundant and repeated entries. Six features, VEC, δ, Δχ, ΔS, ΔH, and Tmelt, are calculated for each data entry. In quantifying the feature significance, different selection techniques were used. With Feature Importance and Correlation based feature selection method (CFS), entropy change is found to be the least important feature. Results from Principal Component Analysis (PCA) and Correlation Analysis suggest that there is no need to drop any feature from the dataset. Nine different datasets were then created with data sampling and fed to four different ML classifiers, where all these classifiers were optimized by tuning the hyperparameters. K-Nearest Neighbor and Random Forest Classifier on the “Oversampled-PCA-6” dataset, with test accuracy of 92.31% and 91.21% respectively, stood as the efficient classifiers for classification of HEAs phases. With above 90% test accuracies, the results from SVM and MLP classifiers were also satisfactory.

Hemoglobin and glucose level estimation from PPG characteristics features of fingertip video using MGGP-based model

Hemoglobin and the glucose level can be measured after taking a blood sample using a needle from the human body and analyzing the sample, the result can be observed. This type of invasive measurement is very painful and uncomfortable for the patient who is required to measure hemoglobin or glucose regularly. However, the non-invasive method only needed a bio-signal (image or spectra) to estimate blood components with the advantages of being painless, cheap, and user-friendliness. In this work, a non-invasive hemoglobin and glucose level estimation model have been developed based on multigene genetic programming (MGGP) using photoplethysmogram (PPG) characteristic features extracted from fingertip video captured by a smartphone. The videos are processed to generate the PPG signal. Analyzing the PPG signal, its first and second derivative, and applying Fourier analysis total of 46 features have been extracted. Additionally, age and gender are also included in the feature set. Then, a correlation-based feature selection method using a genetic algorithm is applied to select the best features. Finally, an MGGP based symbolic regression model has been developed to estimate hemoglobin and glucose level. To compare the performance of the MGGP model, several classical regression models are also developed using the same input condition as the MGGP model. A comparison between MGGP based model and classical regression models have been done by estimating different error measurement indexes. Among these regression models, the best results (±0.304 for hemoglobin and ±0.324 for glucose) are found using selected features and symbolic regression based on MGGP.

Characterisation of prostate cancer using texture analysis for diagnostic and prognostic monitoring.

Automated classification of significant prostate cancer (PCa) using MRI plays a potential role in assisting in clinical decision-making. Multiparametric MRI using a machine-aided approach is a better step to improve the overall accuracy of diagnosis of PCa. The objective of this study was to develop and validate a framework for differentiating Prostate Imaging-Reporting and Data System version 2 (PI-RADS v2) grades (grade 2 to grade 5) of PCa using texture features and machine learning (ML) methods with diffusion-weighted imaging (DWI) and apparent diffusion coefficient (ADC). The study cohort included an MRI dataset of 59 patients with clinically proven PCa. Regions of interest (ROIs) for a total of 435 lesions were delineated from the segmented peripheral zones of DWI and ADC. Six texture methods comprising 98 texture features in total (49 each of DWI and ADC) were extracted from lesion ROIs. Random forest (RF) and correlation-based feature selection methods were applied on feature vectors to select the best features for classification. Two ML classifiers, support vector machine (SVM) and K-nearest neighbour, were used and validated by 10-fold cross-validation. The proposed framework achieved high diagnostic performance with a sensitivity of 85.25% ± 3.84%, specificity of 95.71% ± 1.96%, accuracy of 84.90% ± 3.37% and area under the receiver-operating characteristic curve of 0.98 for PI-RADS v2 grades (2 to 5) classification using the RF feature selection method and Gaussian SVM classifier with combined features of DWI + ADC. The proposed computer-assisted framework can distinguish between PCa lesions with different aggressiveness based on PI-RADS v2 standards using texture analysis to improve the efficiency of PCa diagnostic performance.

Heart disease prediction system using Correlation Based Feature Selection with Multilayer Perceptron approach.

Cardiac disease prediction helps physicians to make accurate recommendations on the treatment of the patients. The use of machine learning (ML) is one of the solution for recognising heart disease-related symptoms. The goal of this study is to suggest a methodology for identifying the most relevant features of cardiac disease characteristics by applying a feature selection technique. The data set used in this study was Framingham heart disease dataset (FHS). It was collected from KAGGLE Machine Learning repository. There are 16 attributes and a mark in the dataset that has been validated by four ML classifiers. There are two feature selection methods, Correlation Based Feature selection (CBFS) and Principle Component Analysis (PCA) was used for the comparison in the study. By using CBFS Method five highly correlated features are selected for the study, and by using PCA thirteen features are selected. The experimental result shows that Correlation Based Feature Selection with Multilayer perceptron (CBFS with MLP) obtained the highest accuracy for this dataset.

Integrative transcriptomic, proteomic, and machine learning approach to identifying feature genes of atrial fibrillation using atrial samples from patients with valvular heart disease

BackgroundAtrial fibrillation (AF) is the most common arrhythmia with poorly understood mechanisms. We aimed to investigate the biological mechanism of AF and to discover feature genes by analyzing multi-omics data and by applying a machine learning approach.MethodsAt the transcriptomic level, four microarray datasets (GSE41177, GSE79768, GSE115574, GSE14975) were downloaded from the Gene Expression Omnibus database, which included 130 available atrial samples from AF and sinus rhythm (SR) patients with valvular heart disease. Microarray meta-analysis was adopted to identified differentially expressed genes (DEGs). At the proteomic level, a qualitative and quantitative analysis of proteomics in the left atrial appendage of 18 patients (9 with AF and 9 with SR) who underwent cardiac valvular surgery was conducted. The machine learning correlation-based feature selection (CFS) method was introduced to selected feature genes of AF using the training set of 130 samples involved in the microarray meta-analysis. The Naive Bayes (NB) based classifier constructed using training set was evaluated on an independent validation test set GSE2240.Results863 DEGs with FDR < 0.05 and 482 differentially expressed proteins (DEPs) with FDR < 0.1 and fold change > 1.2 were obtained from the transcriptomic and proteomic study, respectively. The DEGs and DEPs were then analyzed together which identified 30 biomarkers with consistent trends. Further, 10 features, including 8 upregulated genes (CD44, CHGB, FHL2, GGT5, IGFBP2, NRAP, SEPTIN6, YWHAQ) and 2 downregulated genes (TNNI1, TRDN) were selected from the 30 biomarkers through machine learning CFS method using training set. The NB based classifier constructed using the training set accurately and reliably classify AF from SR samples in the validation test set with a precision of 87.5% and AUC of 0.995.ConclusionTaken together, our present work might provide novel insights into the molecular mechanism and provide some promising diagnostic and therapeutic targets of AF.

Fault diagnosis of tractor auxiliary gearbox using vibration analysis and random forest classifier

Accurate detection of mechanical components faults is an essential step for reduction of repair cost, human injury probability and loss of production. Using intelligent fault diagnosis systems in tractor could prevent secondary damage, thereby avoiding heavy consequences. In this study, fault diagnosis of tractor auxiliary gearbox is presented. Vibration signals of healthy and faulty pinions gear under three different operational conditions (Rotational speeds of 600 RPM, 1350 RPM and 2000 RPM) were collected, and discrete wavelet transform (DWT) was used as signal processing. Useful statistical features were calculated from collected signals. Correlation-based feature selection (CFS) method was used to find the best features. Random forest (RF) and multilayer perceptron (MLP) neural networks were employed to classify the data. The overall accuracy of RF classifier without using feature selection were 86.25%, at 600 RPM. The corresponding values of RF trained with the optimal 6 features by using CFS was 92.5%. The best results obtained at 1350 RPM, since the detection accuracy was 95%. The results of this study demonstrated the effectiveness and feasibility of the proposed method for fault diagnosis of tractor auxiliary gearbox.

Classification of malignant tumours in breast ultrasound using unsupervised machine learning approaches

Traditional computer-aided diagnosis (CAD) processes include feature extraction, selection, and classification. Effective feature extraction in CAD is important in improving the classification’s performance. We introduce a machine-learning method and have designed an analysis procedure of benign and malignant breast tumour classification in ultrasound (US) images without a need for a priori tumour region-selection processing, thereby decreasing clinical diagnosis efforts while maintaining high classification performance. Our dataset constituted 677 US images (benign: 312, malignant: 365). Regarding two-dimensional US images, the oriented gradient descriptors’ histogram pyramid was extracted and utilised to obtain feature vectors. The correlation-based feature selection method was used to evaluate and select significant feature sets for further classification. Sequential minimal optimisation—combining local weight learning—was utilised for classification and performance enhancement. The image dataset’s classification performance showed an 81.64% sensitivity and 87.76% specificity for malignant images (area under the curve = 0.847). The positive and negative predictive values were 84.1 and 85.8%, respectively. Here, a new workflow, utilising machine learning to recognise malignant US images was proposed. Comparison of physician diagnoses and the automatic classifications made using machine learning yielded similar outcomes. This indicates the potential applicability of machine learning in clinical diagnoses.

A Windowed Correlation-Based Feature Selection Method to Improve Time Series Prediction of Dengue Fever Cases

The performance of data-driven models depends on training samples. For accurately predicting dengue fever cases, historical incidence data are inadequate in many locations. This work aims to enhance temporally limited dengue case data by methodological addition of epidemically relevant case data from nearby locations as predictors (features). A novel framework is presented for windowing incidence data and computing time-shifted correlation-based metrics to quantify feature relevance. The framework ranks incidence data of adjacent locations around a target by combining metrics based on correlation, spatial distance, and local prevalence. Recurrent neural network models achieve up to 33.6% accuracy improvement on average using the proposed method. These models achieve mean absolute error (MAE) values as low as 0.128 on [0,1] normalized incidence data for a municipality with the highest dengue prevalence in Brazil’s Espirito Santo. When predicting aggregate cases over geographical ecoregions, the models improve by 16.5%, using only 6.5% of ranked incidence data. This paper also presents two correlation window allocation methods: fixed-size and outbreak detection. Both perform comparably well, although the outbreak detection method uses less data for computations. The proposed framework is generalized, and it can be used to improve time-series predictions of many spatiotemporal datasets.

Heart Disease Prediction and Classification Using Machine Learning Algorithms Optimized by Particle Swarm Optimization and Ant Colony Optimization

The prediction of heart disease is one of the areas where machine learning can be implemented. Optimization algorithms have the advantage of dealing with complex non-linear problems with a good flexibility and adaptability. In this paper, we exploited the Fast Correlation-Based Feature Selection (FCBF) method to filter redundant features in order to improve the quality of heart disease classification. Then, we perform a classification based on different classification algorithms such as K-Nearest Neighbour, Support Vector Machine, Naïve Bayes, Random Forest and a Multilayer Perception | Artificial Neural Network optimized by Particle Swarm Optimization (PSO) combined with Ant Colony Optimization (ACO) approaches. The proposed mixed approach is applied to heart disease dataset; the results demonstrate the efficacy and robustness of the proposed hybrid method in processing various types of data for heart disease classification. Therefore, this study examines the different machine learning algorithms and compares the results using different performance measures, i.e. accuracy, precision, recall, f1-score, etc. A maximum classification accuracy of 99.65% using the optimized model proposed by FCBF, PSO and ACO. The results show that the performance of the proposed system is superior to that of the classification technique presented above.

Feature selection to increase the random forest method performance on high dimensional data

Random Forest is a supervised classification method based on bagging (Bootstrap aggregating) Breiman and random selection of features. The choice of features randomly assigned to the Random Forest makes it possible that the selected feature is not necessarily informative. So it is necessary to select features in the Random Forest. The purpose of choosing this feature is to select an optimal subset of features that contain valuable information in the hope of accelerating the performance of the Random Forest method. Mainly for the execution of high-dimensional datasets such as the Parkinson, CNAE-9, and Urban Land Cover dataset. The feature selection is done using the Correlation-Based Feature Selection method, using the BestFirst method. Tests were carried out 30 times using the K-Cross Fold Validation value of 10 and dividing the dataset into 70% training and 30% testing. The experiments using the Parkinson dataset obtained a time difference of 0.27 and 0.28 seconds faster than using the Random Forest method without feature selection. Likewise, the trials in the Urban Land Cover dataset had 0.04 and 0.03 seconds, while for the CNAE-9 dataset, the difference time was 2.23 and 2.81 faster than using the Random Forest method without feature selection. These experiments showed that the Random Forest processes are faster when using the first feature selection. Likewise, the accuracy value increased in the two previous experiments, while only the CNAE-9 dataset experiment gets a lower accuracy. This research’s benefits is by first performing feature selection steps using the Correlation-Base Feature Selection method can increase the speed of performance and accuracy of the Random Forest method on high-dimensional data.

Performance Enhancement Method for Machine Learning Algorithm

Machine learning is programming computer or a mobile device that learns from experience. Machine learning classification methods are helpful in various fields of Computer Science like driverless cars, product recommendation systems, dynamic pricing, Google translate, online video streaming, internet and mobile fraud detection systems and much more. The present work proposes a method augClassifier to enhance the performance of Simple Logistics machine learning algorithm. The performance assessment of machine learning algorithm is conducted on a Mobile device using Android Environment. The work also presents the comparative performance investigations of Simple Logistics machine learning algorithm using correlation based feature selection method with respect to performance measures Precision, Sensitivity, F-Measure and ROC. The present work conforms that the augClassifier enhances the performance of Simple Logistics machine learning algorithm.

Feature selection for improved classification accuracy targeting riverine sand mapping

Regular monitoring of riverine sand is crucial for its sustainable management. Towards readily generating spatial information regarding the extent of riverine sand, this study utilizes satellite-based remote sensing data as input features while applying Support Vector Machine (SVM) classifier to discern the inherent land cover classes. The input features comprised of spectral bands, derived spectral indices, derived textural features and ancillary information such as elevation, slope and aspect. The objective of this study was to identify a set of features that help improve the SVM classification accuracy with respect to the benchmark accuracy achieved by using only the spectral bands. Apart from testing a few commonly used logical combinations of features, the study focused on employing the Correlation-based Feature Selection (CFS) method along with the best-first search algorithm to generate a feature-set that comprised of most relevant and least correlated features. This feature-set was found to improve the SVM classification accuracy by 2.9% with respect to the benchmark value. Further, when tested with the limited number of training samples, similar results were achieved. This study proves that it is beneficial to utilize the CFS method for feature selection prior to SVM classification and recommends a set of 19 remote sensing derived features that are relevant towards riverine sand mapping.

A robust cyberattack detection approach using optimal features of SCADA power systems in smart grids

Smart grids are a type of complex cyber–physical system (CPS) that integrates the communication capabilities of smart devices into the grid to facilitate remote operation and control of power systems. However, this integration exposes many existing vulnerabilities of conventional supervisory control and data acquisition (SCADA) systems, resulting in severe cyber threats to the smart grid and potential violation of security objectives. Stealing sensitive information, modifying firmware, or injecting function codes through compromised devices are examples of possible attacks on the smart grid. Therefore, early detection of cyberattacks on the grid is crucial to protect it from sabotage. Machine learning (ML) methods are conventional approaches for detecting cyberattacks that use features of smart grid networks. However, developing an effective, highly accurate detection method with reduced computational overload, is still a challenging research problem. In this work, an efficient and effective security control approach is proposed to detect cyberattacks on the smart grid. The proposed approach combines both feature reduction and detection techniques to reduce the extremely large number of features and achieve an improved detection rate. A correlation-based feature selection (CFS) method is used to remove irrelevant features, improving detection efficiency. An instance-based learning (IBL) algorithm classifies normal and cyberattack events using the selected optimal features. This study describes a set of experiments conducted on public datasets from a SCADA power system based on a 10-fold cross-validation technique. Experimental results show that the proposed approach achieves a high detection rate based on a small number of features drawn from SCADA power system measurements.

Feature Selection on Maximum Information Coefficient for Underwater Target Recognition

Feature selection is an essential process in the identification task because the irrelevant and redundant features contained in the unselected feature set can reduce both the performance and efficiency of recognition. However, when identifying the underwater targets based on their radiated noise, the diversity of targets, and the complexity of underwater acoustic channels introduce various complex relationships among the extracted acoustic features. For this problem, this paper employs the normalized maximum information coefficient (NMIC) to measure the correlations between features and categories and the redundancy among different features and further proposes an NMIC based feature selection method (NMIC-FS). Then, on the real-world dataset, the average classification accuracy estimated by models such as random forest and support vector machine is used to evaluate the performance of the NMIC-FS. The analysis results show that the feature subset obtained by NMIC-FS can achieve higher classification accuracy in a shorter time than that without selection. Compared with correlation-based feature selection, laplacian score, and lasso methods, the NMIC-FS improves the classification accuracy faster in the process of feature selection and requires the least acoustic features to obtain classification accuracy comparable to that of the full feature set.

ANCONE: An Interactive System for Mining and Visualization of Students’ Information in the Context of PLANEA 2015

Data mining has been widely used in different areas of knowledge, and education is no exception. Data mining uses computer models to analyze data and answer research questions to help in decision making. This article uses data from the PLANEA 2015 Mathematics in Middle school (last year of Middle school) test, which measures the academic achievement and provides a personal, family and school context, in order to find those characteristics that are related to the academic level of the tested students. In this article, an interactive visualization system was developed that allows observing interesting patterns and association rules by combining relevant attributes (variables) and the States. To reduce the analysis space, the Correlation-Based Feature Selection method was used to reduce categorical and numerical attributes. The results show a significant reduction (93%) in the number of attributes, with very little loss of information, when certain attributes are eliminated. Particularly, the 232 categorical attributes obtained from each student are reduced to only 18 attributes, which are correlated with the results of students in the PLANEA test. In addition, empirically it was discovered that choosing the mode from the labels of plausible values as the target class increases the accuracy in classifiers used to show the goodness of the reduction obtained. Some of the relevant attributes are the "AcademicAspiration", "FamilyResources", "MotherStudies" and "FatherStudies". From the 30 States with information, only 8 are in the Basic level, the other States are in the Below Basic level .

ANCONE: An Interactive System for Mining and Visualization of Students’ Information in the Context of PLANEA 2015

Data mining has been widely used in different areas of knowledge, and education is no exception. Data mining uses computer models to analyze data and answer research questions to help in decision making. This article uses data from the PLANEA 2015 Mathematics in Middle school (last year of Middle school) test, which measures the academic achievement and provides a personal, family and school context, in order to find those characteristics that are related to the academic level of the tested students. In this article, an interactive visualization system was developed that allows observing interesting patterns and association rules by combining relevant attributes (variables) and the States. To reduce the analysis space, the Correlation-Based Feature Selection method was used to reduce categorical and numerical attributes. The results show a significant reduction (93%) in the number of attributes, with very little loss of information, when certain attributes are eliminated. Particularly, the 232 categorical attributes obtained from each student are reduced to only 18 attributes, which are correlated with the results of students in the PLANEA test. In addition, empirically it was discovered that choosing the mode from the labels of plausible values as the target class increases the accuracy in classifiers used to show the goodness of the reduction obtained. Some of the relevant attributes are the AcademicAspiration, FamilyResources, MotherStudies and FatherStudies. From the 30 States with information, only 8 are in the Basic level, the other States are in the Below Basic level .

GIS Based Hybrid Computational Approaches for Flash Flood Susceptibility Assessment

Flash floods are one of the most devastating natural hazards; they occur within a catchment (region) where the response time of the drainage basin is short. Identification of probable flash flood locations and development of accurate flash flood susceptibility maps are important for proper flash flood management of a region. With this objective, we proposed and compared several novel hybrid computational approaches of machine learning methods for flash flood susceptibility mapping, namely AdaBoostM1 based Credal Decision Tree (ABM-CDT); Bagging based Credal Decision Tree (Bag-CDT); Dagging based Credal Decision Tree (Dag-CDT); MultiBoostAB based Credal Decision Tree (MBAB-CDT), and single Credal Decision Tree (CDT). These models were applied at a catchment of Markazi state in Iran. About 320 past flash flood events and nine flash flood influencing factors, namely distance from rivers, aspect, elevation, slope, rainfall, distance from faults, soil, land use, and lithology were considered and analyzed for the development of flash flood susceptibility maps. Correlation based feature selection method was used to validate and select the important factors for modeling of flash floods. Based on this feature selection analysis, only eight factors (distance from rivers, aspect, elevation, slope, rainfall, soil, land use, and lithology) were selected for the modeling, where distance to rivers is the most important factor for modeling of flash flood in this area. Performance of the models was validated and compared by using several robust metrics such as statistical measures and Area Under the Receiver Operating Characteristic (AUC) curve. The results of this study suggested that ABM-CDT (AUC = 0.957) has the best predictive capability in terms of accuracy, followed by Dag-CDT (AUC = 0.947), MBAB-CDT (AUC = 0.933), Bag-CDT (AUC = 0.932), and CDT (0.900), respectively. The proposed methods presented in this study would help in the development of accurate flash flood susceptible maps of watershed areas not only in Iran but also other parts of the world.

Client evaluation decision models in the credit scoring tasks

Abstract One of the important decision-making problems of modern financial institutions is credit scoring, which involves assessing credit risk. Decision-making models based on classifiers and feature selection methods that reduce the complexity of a decision problem by limiting the number of conditional attributes find use in such problems. The article examines the effectiveness of various combinations of classifiers and feature selection methods in the problem of credit risk assessment. The results of the conducted research indicate that for the considered set of data on cash loans granted, the Correlation-based Feature Selection method is the best method among the considered ones, and the Random Forest is the most effective classifier.

Automatic Classification of Microseismic Records in Underground Mining: A Deep Learning Approach

The identification of suspicious microseismic events is the first crucial step in processing microseismic data. In this paper, we present an automatic classification method based on a deep learning approach for classifying microseismic records in underground mines. A total of 35 commonly used features in the time and frequency domains were extracted from waveforms. To examine the discriminative ability of these features, a genetic algorithm (GA)-optimized correlation-based feature selection (CFS) method was applied. As a result, 11 features were selected to represent microseismic records. By dividing each microseismic record into 50 frames, an 11 × 50 feature matrix was utilized as the input. A convolutional neural network (CNN) with 35 layers was trained on 20,000 samples recorded at the Huangtupo Copper and Zinc Mine. There are 5 types of events: microseismic events, blasting, ore extraction, mechanical noise, and electromagnetic interference. The event type was correctly determined by the trained CNN classifier 98.2% of the time, outperforming traditional machine learning methods.

Classification and Prediction of Tibetan Medical Syndrome Based on the Improved BP Neural Network

Tibetan medicine has a long history as a traditional ethnic medicine in China. It is playing an important role in the medical system in northwestern of China, and which has attracted more and more attention due to its unique diagnostic system and clinical efficacy. Meanwhile, as the data mining technology has been widely used in traditional Chinese medicine (TCM), its application in the field of Tibetan medicine has also launched preliminarily. In this paper, we are focusing on Chronic Atrophic Gastritis (CAG) which is a typical gastrointestinal disease in the plateau area, and a novel back-propagation (BP) network model is proposed for Tibetan medical syndrome classification and prediction. K-means clustering algorithm was firstly implemented on the diagnostic data which was obtained from the Qinghai Provincial Tibetan Hospital, and then Correlation-based Feature Selection (CFS) method was adopted for feature selection. The selected feature vectors were finally put into the proposed BP network for training and testing. In order to overcome BP network's typical shortcomings including slow convergence and easy to overfit, we use a method based on Gaussian distribution to improve weights initialization, and dynamically adjusted the learning rate using the learning rate exponential decay method. Further, we add regularization to the loss function to prevent overfitting. Ultimately, the experiment achieved an accuracy of 99.09%, which improved significantly after improvement and achieved better result compared with other classification methods.