Efficient Feature Selection Research Articles

Hybridized artificial neural network classifiers with a novel feature selection procedure based genetic algorithms and information complexity in credit scoring

AbstractThe credit scoring is a statistical analysis performed by financial institutions to represent the creditworthiness of an individual or small and medium‐sized enterprise. A credit score is a numerical quantity that can be qualified by a rating label showing the potential risk. In order to determine which customers are likely to bring in the most revenue at the exact interest rate and credit limits, the lenders take into accounts these credit scores. In this context, the robust statistical models are inevitable to reduce the number of wrong decisions in the credit evaluation process. Although the machine learning approaches provide superior performance to conventional statistical methods, they are mostly criticized due to the selection of model structure, model complexity, tuning parameters, time consumption in the high‐dimensional and excessive nonlinear cases. For this reason, this study introduces an efficient model estimation and feature selection procedure for artificial neural network (ANN) classifiers in the context of credit scoring. Essentially, this procedure hybridizes training of ANNs with a novel feature selection approach based on genetic algorithms and information complexity criterion. In the application, the proposed procedure was performed on a couple of benchmark credit scoring datasets. According to analysis results, the proposed approach not only estimates robust models from ANNs in terms of model complexity, feature selection, and time consumption, but also outperforms the traditional training procedure for the classification accuracies, false positive, and false negative errors overtraining and test datasets.

WITHDRAWN: Coronary heart disease diagnosis using the efficient ANN model

Abstract Heart disease is the foremost among all diseases that widespread over the people community. Nowadays there is no age limit for the people to get affected by this disease. There are so many diagnosis methods available where most are costly and risk involved. The technical experts are needed to perform the disease diagnosis. To carry out the data analytics, the existing diagnosis method uses variety of data classification method, which provides prediction result with less accuracy and more error rate in diagnosing the coronary heart disease stage. This research work focus on enhancing the existing classification with the efficient feature selection method. This reduces the error rate and increases the accuracy of the predictive model. Among all supervised learning algorithm, neural network is found to the efficient model of classification problem. So, this work uses the Neural Network model to intelligently predict the class of heart disease using the efficient features. To prove the correctness of the proposed model, performance is measured. The parameters for performance analysis are precision, sensitivity, specificity, accuracy, error rate. With the results of these parameters, the proposed system is found to be an efficient decision support system for the physician in predicting the stages of coronary heart disease.

Detection and Classification of Bearing Surface Defects Based on Machine Vision

Surface defects on bearings can directly affect the service life and reduce the performance of equipment. At present, the detection of bearing surface defects is mostly done manually, which is labor-intensive and results in poor stability. To improve the inspection speed and the defect recognition rate, we proposed a bearing surface defect detection and classification method using machine vision technology. The method makes two main contributions. It proposes a local multi-neural network (Lc-MNN) image segmentation algorithm with the wavelet transform as the classification feature. The precision segmentation of the defect image is accomplished in three steps: wavelet feature extraction, Lc-MNN region division, and Lc-MNN classification. It also proposes a feature selection algorithm (SCV) that makes comprehensive use of scalar feature selection, correlation analysis, and vector feature selection to first remove similar features through correlation analysis, further screen the results with a scalar feature selection algorithm, and finally select the classification features using a feature vector selection algorithm. Using 600 test samples with three types of defect in the experiment, an identification rate of 99.5% was achieved without the need for large-scale calculation. The comparison tests indicated that the proposed method can achieve efficient feature selection and defect classification.

Enhancing of dataset using DeepDream, fuzzy color image enhancement and hypercolumn techniques to detection of the Alzheimer's disease stages by deep learning model

Alzheimer's disease (AD), which occurs as a result of the loss of cognitive functions in the brain, causes near-forgetfulness in the case and dementia in subsequent processes. Dataset consists of MR images containing four phases of AD. The dataset was re-enhanced separately with DeepDream, fuzzy color image enhancement, hypercolumn techniques. Visual Geometry Group-16 (VGG-16) deep learning model is used in the enhancing process and deep features are combined. Linear Regression is used for the selection of efficient features. The Support Vector Machine is preferred as a classifier. With the proposed approach, the classification achievement was obtained as 100% in Mild Dementia, 99.94% in Moderate Dementia, 100% in non-Dementia, 99.94% in Very Mild Dementia. The overall accuracy was 99.94%. The proposed approach increased the prediction success in detecting Alzheimer's stages by re-enhancing MR images. Thus, an efficient early diagnosis model was realized at an affordable cost for individuals likely to progress with dementia.

Efficient Feature Selection for Static Analysis Vulnerability Prediction.

Common software vulnerabilities can result in severe security breaches, financial losses, and reputation deterioration and require research effort to improve software security. The acceleration of the software production cycle, limited testing resources, and the lack of security expertise among programmers require the identification of efficient software vulnerability predictors to highlight the system components on which testing should be focused. Although static code analyzers are often used to improve software quality together with machine learning and data mining for software vulnerability prediction, the work regarding the selection and evaluation of different types of relevant vulnerability features is still limited. Thus, in this paper, we examine features generated by SonarQube and CCCC tools, to identify those that can be used for software vulnerability prediction. We investigate the suitability of thirty-three different features to train thirteen distinct machine learning algorithms to design vulnerability predictors and identify the most relevant features that should be used for training. Our evaluation is based on a comprehensive feature selection process based on the correlation analysis of the features, together with four well-known feature selection techniques. Our experiments, using a large publicly available dataset, facilitate the evaluation and result in the identification of small, but efficient sets of features for software vulnerability prediction.

Unsupervised Feature Selection With Extended OLSDA via Embedding Nonnegative Manifold Structure.

As to unsupervised learning, most discriminative information is encoded in the cluster labels. To obtain the pseudo labels, unsupervised feature selection methods usually utilize spectral clustering to generate them. Nonetheless, two related disadvantages exist accordingly: 1) the performance of feature selection highly depends on the constructed Laplacian matrix and 2) the pseudo labels are obtained with mixed signs, while the real ones should be nonnegative. To address this problem, a novel approach for unsupervised feature selection is proposed by extending orthogonal least square discriminant analysis (OLSDA) to the unsupervised case, such that nonnegative pseudo labels can be achieved. Additionally, an orthogonal constraint is imposed on the class indicator to hold the manifold structure. Furthermore, l2,1 regularization is imposed to ensure that the projection matrix is row sparse for efficient feature selection and proved to be equivalent to l2,0 regularization. Finally, extensive experiments on nine benchmark data sets are conducted to demonstrate the effectiveness of the proposed approach.

Effective feature representation using symbolic approach for classification and clustering of big data

The tremendous growth in the technology has led to the accumulation of enormous Big Data. Techniques that efficiently analyse this Big Data are in great demand. Tweets from Social media and Sensor data are some of the most common forms of Big Data. Machine learning algorithms pave way for researchers to analyze Big Data. Most Machine learning algorithms depend on efficient feature extraction and feature selection for its success. Here, we explore feature selection methods like entropy and Rough set on the sensor data. Also a symbolic approach of feature extraction is proposed which represents both sensor and twitter data efficiently for further data analysis. Some popular classifiers like Naïve Bayes, K Nearest Neighbour, Support Vector Machine and Decision Tree are used for validating the efficacy of the features selected. An ensemble classifier technique is also proposed which is compared with various state of the art ensemble classifiers. Symbolic features perform better than both entropy and Rough set features for sensor data and improves the clustering efficiency of twitter data. The proposed ensemble weighted average classifier on Symbolic features outperform all the other ensemble classifiers and independent classifiers. The results obtained from these methods have the potential to aid the public health surveillance.

Asymmetric Adaptive Fusion in a Two-Stream Network for RGB-D Human Detection.

In recent years, human detection in indoor scenes has been widely applied in smart buildings and smart security, but many related challenges can still be difficult to address, such as frequent occlusion, low illumination and multiple poses. This paper proposes an asymmetric adaptive fusion two-stream network (AAFTS-net) for RGB-D human detection. This network can fully extract person-specific depth features and RGB features while reducing the typical complexity of a two-stream network. A depth feature pyramid is constructed by combining contextual information, with the motivation of combining multiscale depth features to improve the adaptability for targets of different sizes. An adaptive channel weighting (ACW) module weights the RGB-D feature channels to achieve efficient feature selection and information complementation. This paper also introduces a novel RGB-D dataset for human detection called RGBD-human, on which we verify the performance of the proposed algorithm. The experimental results show that AAFTS-net outperforms existing state-of-the-art methods and can maintain stable performance under conditions of frequent occlusion, low illumination and multiple poses.

Human gait analysis for osteoarthritis prediction: a framework of deep learning and kernel extreme learning machine

Human gait analysis is a novel topic in the field of computer vision with many famous applications like prediction of osteoarthritis and patient surveillance. In this application, the abnormal behavior like problems in walking style is detected of suspected patients. The suspected behavior means assessments in terms of knee joints and any other symptoms that directly affected patients’ walking style. Human gait analysis carries substantial importance in the medical domain, but the variability in patients’ clothes, viewing angle, and carrying conditions, may severely affect the performance of a system. Several deep learning techniques, specifically focusing on efficient feature selection, have been recently proposed for this purpose, unfortunately, their accuracy is rather constrained. To address this disparity, we propose an aggregation of robust deep learning features in Kernel Extreme Learning Machine. The proposed framework consists of a series of steps. First, two pre-trained Convolutional Neural Network models are retrained on public gait datasets using transfer learning, and features are extracted from the fully connected layers. Second, the most discriminant features are selected using a novel probabilistic approach named Euclidean Norm and Geometric Mean Maximization along with Conditional Entropy. Third, the aggregation of the robust features is performed using Canonical Correlation Analysis, and the aggregated features are subjected to various classifiers for final recognition. The evaluation of the proposed scheme is performed on a publicly available gait image dataset CASIA B. We demonstrate that the proposed feature aggregation methodology, once used with the Kernel Extreme Learning Machine, achieves accuracy beyond 96%, and outperforms the existing works and several other widely adopted classifiers.

Motor Imagery EEG Spectral-Spatial Feature Optimization Using Dual-Tree Complex Wavelet and Neighbourhood Component Analysis

BackgroundFrequency band optimization improves the performance of common spatial pattern (CSP) in motor imagery (MI) tasks classification because MI-related electroencephalograms (EEGs) are highly frequency specific. Many variants of CSP algorithm divided the EEG into various sub bands and then applied CSP. However, the feature dimension of MI-EEG data increases with addition of frequency sub bands and requires efficient feature selection algorithms. The performance of CSP also depends on filtering techniques. MethodIn this study, we designed a dual tree complex wavelet transform based filter bank to filter the EEG into sub bands, instead of traditional filtering methods, which improved the spatial feature extraction efficiency. Further, after filtering EEG into different sub bands, we extracted spatial features from each sub band using CSP and optimized them by a proposed supervised learning framework based on neighbourhood component analysis (NCA). Subsequently, a support vector machine (SVM) is trained to perform classification. ResultsAn experimental study, conducted on two datasets (BCI Competition IV (Dataset 2b), and BCI competition III (Dataset IIIa)), validated the MI classification effectiveness of the proposed method in comparison with standard algorithms such as CSP, Filter bank CSP (CSP), and Discriminative FBCSP (DFBCSP). The average classification accuracy obtained by the proposed method for BCI Competition IV (Dataset 2b), and BCI Competition III (Dataset IIIa) are 84.02 ± 12.2 and 89.1 ± 7.50, respectively and found significant than that achieved by standard methods. ConclusionAchieved superior results suggest that the proposed algorithm can improve the performance of MI-based Brain-computer interface devices.

Two-stage approach to feature set optimization for unsupervised dataset with heterogeneous attributes

Unsupervised feature selection (UFS) is utilized in various application domains, such as data mining, pattern recognition, machine learning, etc. UFS follows three basic approaches, namely filter, wrapper, and hybrid (that is, a combination of both filter and wrapper) to select the relevant and non-redundant features. It has been observed that a filter method does not guarantee an optimal solution. However, a wrapper approach is computationally expensive. The hybrid method are known to give a better trade-off between filter and wrapper strategies. But, the practical applicability of schemes mentioned above are preferably restricted only to a numerical dataset and are not so suitable for a mixed dataset. Therefore, there is a need for a UFS scheme which can handle both the numerical and non-numerical features directly. In this paper, a robust and efficient two-phase (i.e., feature ranking (FR) and feature selection (FS)) UFS method is proposed. The proposed FR utilizes entropy and mutual information to produce maximum informative and non-redundant ranked features from a high-dimensional mixed dataset. Further, the proposed FS follows k-prototype clustering algorithm with improved Callinski-Harasbaz criteria-based selection methodology to choose optimal features. Experiments on real-life dataset substantiate that the proposed approach provides a better subset of features compared to the existing state of the art approaches.

A Novel Multi-Stage Training Approach for Human Activity Recognition From Multimodal Wearable Sensor Data Using Deep Neural Network

Deep neural network is an effective choice to automatically recognize human actions utilizing data from various wearable sensors. These networks automate the process of feature extraction relying completely on data. However, various noises in time series data with complex inter-modal relationships among sensors make this process more complicated. In this article, we have proposed a novel multi-stage training approach that increases diversity in this feature extraction process to make accurate recognition of actions by combining varieties of features extracted from diverse perspectives. Initially, instead of using single type of transformation, numerous transformations are employed on time series data to obtain variegated representations of the features encoded in raw data. An efficient deep CNN architecture is proposed that can be individually trained to extract features from different transformed spaces. Later, these CNN feature extractors are merged into an optimal architecture finely tuned for optimizing diversified extracted features through a combined training stage or multiple sequential training stages. This approach offers the opportunity to explore the encoded features in raw sensor data utilizing multifarious observation windows with immense scope for efficient selection of features for final convergence. Extensive experimentations have been carried out in three publicly available datasets that provide outstanding performance consistently with average five-fold cross-validation accuracy of 99.29% on UCI HAR database, 99.02% on USC HAR database, and 97.21% on SKODA database outperforming other state-of-the-art approaches.

Efficient feature selection and classification through ensemble method for network intrusion detection on cloud computing

Cloud computing is a preferred option for organizations around the globe, it offers scalable and internet-based computing resources as a flexible service. Security is a key concern factor in any cloud solution due to its distributed nature. Security and privacy are huge obstacles faced in its success of the on-demand service as it is easily vulnerable to intruders for any kind of attack. A huge upsurge in network traffic has paved the way to security breaches which are more complicated and widespread. Tackling these attacks has become an inefficient application of traditional intrusion detection systems (IDS) environment. In this research, we developed an efficient Intrusion Detection System (IDS) for the cloud environment using ensemble feature selection and classification techniques. This proposed method was relying on the univariate ensemble feature selection technique, which is used for the selection of valuable reduced feature sets from the given intrusion datasets. While the ensemble classifiers that can competently fuse the single classifiers to produce a robust classifier using the voting technique. An ensemble based proposed method effectively classifies whether the network traffic behavior is normal or attack. The implementation of the proposed method was measured by applying various performance evaluation metrics and ROC-AUC (“area under the receiver operating characteristic curves”) across various classifiers. The results of the proposed methodology achieved a strong considerable amount of performance enhancement compared with other existing methods. Moreover, we performed a pairwise t test and proved that the performance of the proposed method was statistically significantly different from other existing approaches. Finally, the outcome of this investigation was obtained with the best accuracy and lowest false alarm rate (FAR).

Efficient Features for Function Matching in Multi-Architecture Binary Executables

Binary-binary function matching problem serves as a plinth in many reverse engineering techniques such as binary diffing, malware analysis, and code plagiarism detection. In literature, function matching is performed by first extracting function features (syntactic and semantic), and later these features are used as selection criteria to formulate an approximate 1:1 correspondence between binary functions. The accuracy of the approximation is dependent on the selection of efficient features. Although substantial research has been conducted on this topic, we have explored two major drawbacks in previous research. (i) The features are optimized only for a single architecture and their matching efficiency drops for other architectures. (ii) function matching algorithms mainly focus on the structural properties of a function, which are not inherently resilient against compiler optimizations. To resolve the architecture dependency and compiler optimizations, we benefit from the intermediate representation (IR) of function assembly and propose a set of syntactic and semantic (embedding-based) features which are efficient for multi-architectures, and sensitive to compiler-based optimizations. The proposed function matching algorithm employs one-shot encoding that is flexible to small changes and uses a KNN based approach to effectively map similar functions. We have evaluated proposed features and algorithms using various binaries, which were compiled for ×86 and ARM architectures; and the prototype implementation is compared with Diaphora (an industry-standard tool), and other baseline research. Our proposed prototype has achieved a matching accuracy of approx. 96%, which is higher than the compared tools and consistent against optimizations and multi-architecture binaries.

Time-Frequency Analysis of EEG Signals and GLCM Features for Depth of Anesthesia Monitoring.

One of the important tasks in the operating room is monitoring the depth of anesthesia (DoA) during surgery, and noninvasive techniques are very popular. Hence, we propose a new scheme for DoA monitoring considering the time-frequency analysis of electroencephalography (EEG) signals and GLCM features extracted from them. To this end, at first, the time-frequency map (TFM) of each channel of each EEG is computed by smoothed pseudo-Wigner–Ville distribution (SPWVD), where the EEG signal used in this paper is recorded in 15 channels. After that, we consider the gray-level co-occurrence matrix (GLCM) to obtain the content of TFM, and after that, four features such as homogeneity, correlation, energy, and contrast are obtained for each GLCM. Finally, after the selection of efficient features using the minimum redundancy maximum relevance (MRMR) method, the K-nearest neighbor (KNN) classifier is utilized to determine the DoA. Here, we consider the three states, namely, deep hypnotic, surgical anesthesia, and sedation and awake states according to bispectral index (BIS), and each EEG epoch is classified to these states. We also employ data augmentation to enhance the training phase and increase accuracy. We obtain the accuracy and confusion matrix of the proposed method. We also analyze the effects of a number of gray levels of GLCM, distance measure in KNN classifier, and parameters of data augmentation on the performance of the proposed method. Results indicate the efficiency of the proposed method to determine the DoA during surgery.

Feature selection using efficient fusion of Fisher Score and greedy searching for Alzheimer’s classification

Alzheimer’s disease is one of the deadly progressive neurodegenerative diseases among aged populations. But, the progression of the disease can be reduced by proper treatment of the disease during the early stages of cognitive impairment. The main objective of this study is to implement an efficient feature selection algorithm for the detection of Alzheimer’s patients at the baseline stage itself using multimodal data. In this paper, we propose an efficient fusion of Fisher Score ranking and greedy searching heuristic as feature selection criteria for Alzheimer’s prediction. The proposed algorithm provides a Balanced Classification Accuracy of 90% and 91% and Multi Area Under the Curve of 0.97, 0.98 using Support Vector Machine, K-Nearest Neighbor respectively for classifying Normal Controls, Mild Cognitive Impairment, and Alzheimer’s patients on Alzheimer’s Disease Neuroimaging Initiative-TADPOLE dataset at baseline visit itself. Moreover, the proposed algorithm also provides better sensitivity, specificity of 84%, 82.5% using Support Vector Machine, K-Nearest Neighbor for binary classification of Mild Cognitive Impairment, and Alzheimer’s patients on the Australian Imaging and Biomarker Lifestyle dataset also. Our results indicate that the proposed methodology with efficient feature selection is promising and can outperform the state of the art methods for early detection of Alzheimer’s.

Stable Hybrid Feature Selection Method for Compressor Fault Diagnosis

Faulty compressors must be detected in advance to speed up the quality control process of the compressor's performance. Machine learning models have recently been used as fault classification models to distinguish between normal and abnormal compressors, facilitating more sophisticated fault detection methods than those in the past. However, very few studies have been conducted on accurate and efficient feature selection, despite its high importance. Therefore, this study proposes a new hybrid method that combines the merits of existing feature methods, filter and wrapper methods, to obtain a stable, accurate, and efficient fault classification model. For this, three types of filtering methods with different characteristics, such as chi-square, extra tree classifier, and correlation matrix, are used to derive the high-ranked features and then create a powerful feature set consisting of their union sets. Subsequently, using the wrapper method, one combination of features with the highest classification accuracy was selected among all the combinations of features in the union feature set. Using two experimental examples and one numerical example with different types and numbers of data, the robustness and accuracy of the proposed method were verified through comparison with the existing filter methods by combining three classification models: support vector machine, K-nearest neighbor, and multi-layer perceptron.

An Efficient SVM-Based Feature Selection Model for Cancer Classification Using High-Dimensional Microarray Data

An Efficient SVM-Based Feature Selection Model for Cancer Classification Using High-Dimensional Microarray Data

An Efficient Fingertip Photoplethysmographic Signal Artifact Detection Method: A Machine Learning Approach

A photoplethysmography method has recently been widely used to noninvasively measure blood volume changes during a cardiac cycle. Photoplethysmogram (PPG) signals are sensitive to artifacts that negatively impact the accuracy of many important measurements. In this paper, we propose an efficient system for detecting PPG signal artifacts acquired from a fingertip in the public healthcare database named Multiparameter Intelligent Monitoring in Intensive Care (MIMIC) by using 11 features as the input of the random forest algorithm and classified the signals into two classes: acceptable and anomalous. A real‐time algorithm is proposed to identify artifacts by using the method. The efficient Fisher score feature selection algorithm was used to order and select 11 relevant features from 19 available features that represented the PPG signal very effectively. Six machine learning algorithms (random forest, decision tree, Gaussian naïve Bayes, linear support vector machine, artificial neural network, and probabilistic neural network) were applied with the extracted features, and their classification accuracy was measured. Among them, the random forest had the best performance using only 11 out of 19 features with an accuracy of 85.68%. Our proposed method also achieved good sensitivity and specificity value of 86.57% and 85.09%, respectively. The proposed real‐time algorithm can be an easy and convenient way for real‐time PPG signal artifact detection using smartphones and wearable devices.

Data-Driven Predictive Control of Building Energy Consumption under the IoT Architecture

Model predictive control is theoretically suitable for optimal control of the building, which provides a framework for optimizing a given cost function (e.g., energy consumption) subject to constraints (e.g., thermal comfort violations and HVAC system limitations) over the prediction horizon. However, due to the buildings’ heterogeneous nature, control-oriented physical models’ development may be cost and time prohibitive. Data-driven predictive control, integration of the “Internet of Things”, provides an attempt to bypass the need for physical modeling. This work presents an innovative study on a data-driven predictive control (DPC) for building energy management under the four-tier building energy Internet of Things architecture. Here, we develop a cloud-based SCADA building energy management system framework for the standardization of communication protocols and data formats, which is favorable for advanced control strategies implementation. Two DPC strategies based on building predictive models using the regression tree (RT) and the least-squares boosting (LSBoost) algorithms are presented, which are highly interpretable and easy for different stakeholders (end-user, building energy manager, and/or operator) to operate. The predictive model’s complexity is reduced by efficient feature selection to decrease the variables’ dimensionality and further alleviate the DPC optimization problem’s complexity. The selection is dependent on the principal component analysis (PCA) and the importance of disturbance variables (IoD). The proposed strategies are demonstrated both in residential and office buildings. The results show that the DPC-LSBoost has outperformed the DPC-RT and other existing control strategies (MPC, TDNN) in performance, scalability, and robustness.