Information Feature Selection Research Articles

Determinant of Covariance Matrix Model Coupled with AdaBoost Classification Algorithm for EEG Seizure Detection.

Experts usually inspect electroencephalogram (EEG) recordings page-by-page in order to identify epileptic seizures, which leads to heavy workloads and is time consuming. However, the efficient extraction and effective selection of informative EEG features is crucial in assisting clinicians to diagnose epilepsy accurately. In this paper, a determinant of covariance matrix (Cov–Det) model is suggested for reducing EEG dimensionality. First, EEG signals are segmented into intervals using a sliding window technique. Then, Cov–Det is applied to each interval. To construct a features vector, a set of statistical features are extracted from each interval. To eliminate redundant features, the Kolmogorov–Smirnov (KST) and Mann–Whitney U (MWUT) tests are integrated, the extracted features ranked based on KST and MWUT metrics, and arithmetic operators are adopted to construe the most pertinent classified features for each pair in the EEG signal group. The selected features are then fed into the proposed AdaBoost Back-Propagation neural network (AB_BP_NN) to effectively classify EEG signals into seizure and free seizure segments. Finally, the AB_BP_NN is compared with several classical machine learning techniques; the results demonstrate that the proposed mode of AB_BP_NN provides insignificant false positive rates, simpler design, and robustness in classifying epileptic signals. Two datasets, the Bern–Barcelona and Bonn datasets, are used for performance evaluation. The proposed technique achieved an average accuracy of 100% and 98.86%, respectively, for the Bern–Barcelona and Bonn datasets, which is considered a noteworthy improvement compared to the current state-of-the-art methods.

Authorship Attribution of Social Media and Literary Russian-Language Texts Using Machine Learning Methods and Feature Selection

Authorship attribution is one of the important fields of natural language processing (NLP). Its popularity is due to the relevance of implementing solutions for information security, as well as copyright protection, various linguistic studies, in particular, researches of social networks. The article is a continuation of the series of studies aimed at the identification of the Russian-language text’s author and reducing the required text volume. The focus of the study was aimed at the attribution of textual data created as a product of human online activity. The effectiveness of the models was evaluated on the two Russian-language datasets: literary texts and short comments from users of social networks. Classical machine learning (ML) algorithms, popular neural networks (NN) architectures, and their hybrids, including convolutional neural network (CNN), networks with long short-term memory (LSTM), Bidirectional Encoder Representations from Transformers (BERT), and fastText, that have not been used in previous studies, were applied to solve the problem. A particular experiment was devoted to the selection of informative features using genetic algorithms (GA) and evaluation of the classifier trained on the optimal feature space. Using fastText or a combination of support vector machine (SVM) with GA reduced the time costs by half in comparison with deep NNs with comparable accuracy. The average accuracy for literary texts was 80.4% using SVM combined with GA, 82.3% using deep NNs, and 82.1% using fastText. For social media comments, results were 66.3%, 73.2%, and 68.1%, respectively.

Computer‐aided diagnosis ofCOVID‐19 disease from chest X‐ray images integrating deep feature extraction

AbstractThe novel coronavirus (COVID‐19) has an enormous impact on the daily lives and health of people residing in more than 200 nations. This article proposes a deep learning‐based system for the rapid diagnosis of COVID‐19. Chest x‐ray radiograph images were used because recent findings revealed that these images contain salient features about COVID‐19 disease. Transfer learning was performed using different pre‐trained convolutional neural networks models for binary (normal and COVID‐19) and triple (normal, COVID‐19 and viral pneumonia) class problems. Deep features were extracted from a fully connected layer of the ResNET50v2 model and feature dimension was reduced through feature reduction methods. Feature fusion of feature sets reduced through analysis of variance (ANOVA) and mutual information feature selection (MIFS) was fed to Fine K‐nearest neighbour to perform binary classification. Similarly, serial feature fusion of MIFS and chi‐square features were utilized to train Medium Gaussian Support Vector Machines to distinguish normal, COVID‐19 and viral pneumonia cases. The proposed framework yielded accuracies of 99.5% for binary and 95.5% for triple class experiments. The proposed model shows better performance than the existing methods, and this research has the potential to assist medical professionals to enhance the diagnostic ability to detect coronavirus disease.

STUDY NEUR NETWORKS FOR SOFTWARE DEFINED RADIO CONTROL

The scientific article is devoted to the issues of SDR system control. Software Defined Radio is a system designed for software control of information transmission processes in a radio communication channel. Recognition of digital modulation types is used, which automatically classifies the type of digital modulation of the received signal. The following issues are covered in the article: the analysis of existing approaches in the task of automatic recognition of types of digital modulation is carried out; the analysis and classification of informative features in the task of automatic recognition is carried out the following types of digital modulation: 2-PSK, 4-PSK, 8-PSK, 2-FSK, 8-QAM, 16-QAM, 64-QAM and OFDM. This article uses a neural network approach based on cumulative characteristics. To solve these problems, the methods of calculation and selection of informative cumulative features described in the developed model of the system of automatic recognition of digital modulation types by means of a neural network on cumulative features at a known value of carrier frequency are described. A model of the system of automatic recognition of digital modulation types at a known value is constructed carrier frequency. An algorithm for automatic recognition of digital modulation types has been developed using a multilayer neural network. The influence of noise in the communication channel on the probability of recognizing the types of digital modulation at a known value of the carrier frequency is investigated. It was found that regardless of the type of noise in the communication channel, the law of error distribution in IQ data becomes close to normal. This fact is one important argument for the use of cumulative features in the task of automatic recognition of types of digital modulation. Therefore, the task of automatic recognition of digital modulation types is quite relevant. Further research may be aimed at expanding the range of high-order cumulative features used, due to which it is possible to increase the probability of correct recognition of types of digital modulation, and solving the recognition problem at an unknown value of the frequency and initial phase of the carrier signal.

Feature selection and classification over the network with missing node observations.

Jointly analyzing transcriptomic data and the existing biological networks can yield more robust and informative feature selection results, as well as better understanding of the biological mechanisms. Selecting and classifying node features over genome-scale networks has become increasingly important in genomic biology and genomic medicine. Existing methods have some critical drawbacks. The first is they do not allow flexible modeling of different subtypes of selected nodes. The second is they ignore nodes with missing values, very likely to increase bias in estimation. To address these limitations, we propose a general modeling framework for Bayesian node classification (BNC) with missing values. A new prior model is developed for the class indicators incorporating the network structure. For posterior computation, we resort to the Swendsen-Wang algorithm for efficiently updating class indicators. BNC can naturally handle missing values in the Bayesian modeling framework, which improves the node classification accuracy and reduces the bias in estimating gene effects. We demonstrate the advantages of our methods via extensive simulation studies and the analysis of the cutaneous melanoma dataset from The Cancer Genome Atlas.

Survival analysis with semi-supervised predictive clustering trees

Many clinical studies follow patients over time and record the time until the occurrence of an event of interest (e.g., recovery, death, …). When patients drop out of the study or when their event did not happen before the study ended, the collected dataset is said to contain censored observations. Given the rise of personalized medicine, clinicians are often interested in accurate risk prediction models that predict, for unseen patients, a survival profile, including the expected time until the event. Survival analysis methods are used to detect associations or compare subpopulations of patients in this context.In this article, we propose to cast the time-to-event prediction task as a multi-target regression task, with censored observations modeled as partially labeled examples. We then apply semi-supervised learning to the resulting data representation. More specifically, we use semi-supervised predictive clustering trees and ensembles thereof.Empirical results over eleven real-life datasets demonstrate superior or equivalent predictive performance of the proposed approach as compared to three competitor methods. Moreover, smaller models are obtained compared to random survival forests, another tree ensemble method. Finally, we illustrate the informative feature selection mechanism of our method, by interpreting the splits induced by a single tree model when predicting survival for amyotrophic lateral sclerosis patients.

Identification of argumentative sentences in Russian scientific and popular science texts

In this study we analyze the applicability of specific machine learning algorithms to the task of detecting sentences containing argumentation in Russian text. We employ a collection of scientific and popular science texts with manually annotated argumentation to evaluate the quality of identifying argumentative sentences in terms of precision, recall, and F-measure. The experiment involves three algorithms: MNB, SVM, and MLP. The bag of words model is used for representing texts. Lemmas of words in analyzed sentences serve as features for the classification. We perform the automatic selection of informative features in accordance with Variance and χ2 criteria combined with the weight-based filtration of lemmas (via TF*IDF and EMI). The training set includes around 800 sentences, while the test set contains 180. The MNB algorithm demonstrates the highest F-measure and recall scores on almost all feature sets (maximal values reached equal 68.7% and 89% respectively), while the MLP algorithm shows the best precision for about half of feature selection variations (the maximal value is 72.5%).

Manifold Feature Fusion with Dynamical Feature Selection for Cross-Subject Emotion Recognition.

Affective computing systems can decode cortical activities to facilitate emotional human–computer interaction. However, personalities exist in neurophysiological responses among different users of the brain–computer interface leads to a difficulty for designing a generic emotion recognizer that is adaptable to a novel individual. It thus brings an obstacle to achieve cross-subject emotion recognition (ER). To tackle this issue, in this study we propose a novel feature selection method, manifold feature fusion and dynamical feature selection (MF-DFS), under transfer learning principle to determine generalizable features that are stably sensitive to emotional variations. The MF-DFS framework takes the advantages of local geometrical information feature selection, domain adaptation based manifold learning, and dynamical feature selection to enhance the accuracy of the ER system. Based on three public databases, DEAP, MAHNOB-HCI and SEED, the performance of the MF-DFS is validated according to the leave-one-subject-out paradigm under two types of electroencephalography features. By defining three emotional classes of each affective dimension, the accuracy of the MF-DFS-based ER classifier is achieved at 0.50–0.48 (DEAP) and 0.46–0.50 (MAHNOBHCI) for arousal and valence emotional dimensions, respectively. For the SEED database, it achieves 0.40 for the valence dimension. The corresponding accuracy is significantly superior to several classical feature selection methods on multiple machine learning models.

Identification of most important features based on a fuzzy ensemble technique: Evaluation on joint space narrowing progression in knee osteoarthritis patients

ObjectiveFeature selection (FS) is a crucial and at the same time challenging processing step that aims to reduce the dimensionality of complex classification or regression problems. Various techniques have been proposed in the literature to address this challenge with emphasis to medical applications. However, each one of the existing FS algorithms come with its own advantages and disadvantages introducing a certain level of bias. Materials and MethodsTo avoid bias and alleviate the defectiveness of single feature selection results, an ensemble FS methodology is proposed in this paper that aggregates the results of several FS algorithms (filter, wrapper and embedded ones). Fuzzy logic is employed to combine multiple feature importance scores thus leading to a more robust selection of informative features. The proposed fuzzy ensemble FS methodology was applied on the problem of knee osteoarthritis (KOA) prediction with special emphasis on the progression of joint space narrowing (JSN). The proposed FS methodology was integrated into an end-to-end machine learning pipeline and a thorough experimental evaluation was conducted using data from the Osteoarthritis Initiative (OAI) database. Several classifiers were investigated for their suitability in the task of JSN prediction and the best performing model was then post-hoc analyzed by using the SHAP method. ResultsThe results showed that the proposed method presented a better and more stable performance in contrast to other competitive feature selection methods, leading to an average accuracy of 78.14% using XG Boost at 31 selected features. The post-hoc explainability highlighted the important features that contribute to the classification of patients with JSN progression. ConclusionsThe proposed fuzzy feature selection approach improves the performance of the predictive models by selecting a small optimal subset of features compared to popular feature selection methods.

Integration of aggressive bound tightening and Mixed Integer Programming for Cost-sensitive feature selection in medical diagnosis

Silent diseases is an umbrella term that captures a spectrum of chronic illnesses that produce no clinically obvious signs and are diagnosed at advanced stages when the damage is irreversible. Current diagnostic strategies of silent diseases depend on self-reported symptoms and observed behavior through extended periods of time, and until now there are no specific clinical tests to diagnose silent diseases. Scientific research suggests the importance of early diagnosis to restore the functionality and reduce diseases-related complications. Previous studies primarily focused on feature selection methods to aid in medical diagnosis. Traditional feature selection methods are primarily focused on correct classification and often ignore features’ costs; the cost of clinical tests required to acquire the feature value. However, in medical diagnosis, features have different associated costs. Because ignoring features’ costs may result in a high cost diagnostic strategy that cannot be used in practice, developing a low-cost diagnostic strategy remains a subject of much interest. In this paper, new Mixed Integer Programming (MIP) models, namely, Cost-sensitive Support Vector Machine (CS-SVM) and Cost-sensitive Multi-surface Method Tree (CS-MSMT) that allow for simultaneous selection of low-cost and informative features are proposed. The CS-SVM and CS-MSMT are superior because they have the ability to account for shared costs. The CS-SVM and CS-MSMT were modified to embed shared costs across feature groups, and are termed Discounted CS-SVM (dCS-SVM) and Discounted CS-MSMT (dCS-MSMT), respectively. Computationally effective algorithm that integrates aggressive bound tightening with the MIP formulation is proposed. To demonstrate the effectiveness of the proposed models, different analysis paradigms are conducted on six UCI medical datasets; Chronic Kidney Disease, Hepatitis, Heart Disease, Thyroid, Diabetes and Leukemia. The results demonstrate the efficiency and robustness of the CS-SVM and CS-MSMT (and consequently the dCS-SVM and dCS-MSMT) under various conditions. The CS-SVM and CS-MSMT improved accuracy by 10.3% and 3.4% and reduced costs by 94.3% and 72.4% in the leukemia dataset, respectively.

Machine-learned security assessment for changing system topologies

Machine learning has been used in the past to construct predictors, also known as classifiers, for dynamic security assessment. Although accurate classifiers can be trained for a single topology, often they do not work for another. However, the power system topology can change frequently during operation due to maintenance and control actions. At one topological configuration, the system may have a different response to a fault than at another as the underlying distribution of power flows can be completely different. Quantifying the impact of changes in the topology on the predictive models’ performance is an important step forward to minimize inaccurate predictions and improve their reliability.In this paper, for the first time, a metric for quantifying the impact of a topology change on the accuracy of the classification model is proposed. The key novelty is to first select a subset of power flow features with a physically informed feature selection technique and subsequently compute the metric with a novel convex hull-based analysis. In addition, the approach can advise to effectively constructing new training databases that improve the accuracy of new machines trained after high-impact topology changes. Through a case study using transient stability on the IEEE 68-bus system, the use of the proposed metric in real-time operation was demonstrated. 17 high-impact topology changes were successfully detected among 42 studied topological changes. The subsequent effective construction of the training database improved the predictive accuracy by around 10%. An interesting finding is the amount of newly generated data can be reduced by up to 85% as often the generated data is the barrier for data-driven DSA. The proposed workflow significantly reduces data and trains robust classifiers against topological changes marking a fundamental step forward.

Analyzing the Features Affecting the Performance of Teachers during Covid-19: A Multilevel Feature Selection

COVID-19 is a profoundly contagious pandemic that has taken the world by storm and has afflicted different fields of life with negative effects. It has had a substantial impact on education which is evident from the transition from Face-to-Face (F2F) teaching to online mode of education and the rigid implementation of lockdown across the globe. This study examines the factors impacting the performance of teachers during the lockdown period of COVID-19 using various feature selection algorithms and Artificial Intelligence techniques. In this paper, we have proposed a novel multilevel feature selection for the prediction of the factors affecting the teachers’ satisfaction with online teaching and learning in COVID-19. The proposed multilevel feature selection is composed of the Fast Correlation Based Filter (FCBF), Mutual Information (MI), Relieff, and Particle Swarm Optimization (PSO) feature selection. The performance of the proposed feature selection approach is evaluated through the teachers’ benchmark dataset. We used a range of measures like accuracy, precision, f-measure, and recall to evaluate the performance of the proposed approach. We applied different machine learning approaches (SVM, LGBM, and ANN) with the proposed multilevel feature selection technique. The performance of the proposed approach is also compared with existing feature selection algorithms, and the results show the improvement in the performance of feature selection in terms of accuracy, precision, recall, and F-Measure. Proposed feature selection provides more than 80% accuracy with Light Weight Gradient Boosting Machine (LGBM).

Informative Features Selection for Building an Optimization Model of the Aluminum Electrolytic Cell Thermal Regime

Mathematical modeling is the main tool for optimizing the operation of aluminum electrolytic cells. It helps to get answers to important questions to developers of complex systems in the aluminum industry. It contributes to the intensification and improvement of these systems development in the quality with minimal resources and time. One of the main parameters characterizing the process of aluminum electrolysis is the current efficiency, an output indicator that evaluates the efficiency of the aluminum electrolytic cells. The paper proposes to use modeling regression equations for automatic control of the electrolyte temperature according to the statistical parameters of voltage fluctuations. It helps to control the technological process of electrolysis with correction for the melt temperature quickly. The authors propose a mathematical model of the thermal regime of an aluminum electrolytic cell based on an estimation of the relationship between the melt temperature and the main indicators of the electrolysis process (melt temperature and composition, concentration of dissolved alumina, cryolite ratio, anode-cathode-distance (ACD) and voltage fluctuation parameters.

Building a Fuzzy Classifier Based on Whale Optimization Algorithm to Detect Network Intrusions

The quantity of network attacks and the harm from them is constantly increasing, so the detection of these attacks is an urgent task in the information security field. In this paper, we investigate an approach to building intrusion detection systems using a classifier based on fuzzy rules. The process of creating a fuzzy classifier based on a given set of input and output data can be presented as a solution to the problems of clustering, informative features selection, and the parameters of the rule antecedents optimization. To solve these problems, the whale optimization algorithm is used. The performance of algorithms for constructing a fuzzy classifier based on this metaheuristic is estimated using the KDD Cup 1999 intrusion detection dataset. On average, the resulting classifiers have a type I error of 0.92% and a type II error of 1.07%. The obtained results are also compared with the results of other classifiers. The comparison shows the competitiveness of the proposed method.

Probabilistic decision support system using machine learning techniques : A case study of Cardiovascular diseases

Diseases are an irregular event that affects single or multiple body parts of a human being. Different kinds of diseases are growing day by day in terms of lifestyle and heritage. Among all the ones, it turns out that heart disease is the most prevalent disease, and the consequence of this disease is dangerous relative to all other situations. In this paper, initially the author calculates immanent discreteness between two probability separations, based on information theory or relative entropy using Kullback-Leibler divergence (KL divergence). It can be used as shortcuts in the calculation of other widely used methods, such as mutual information for feature selection before modeling, and cross-entropy used as a loss function for many different classifier models. Seven computational intelligence techniques named logistic regression (LR), support vector machine (SVM), naïve bayes (NB), random forest (RF), and k-nearest neighbor (K-NN) have been applied and a comparative study had been drawn. The heart disease dataset is taken from the Department of Cardiology, Excelcare Hospitals, Guwahati, Assam and the analysis of performance in each technique has been evaluated.

Mutual Information Feature Selection (MIFS) Based Crop Yield Prediction on Corn and Soybean Crops Using Multilayer Stacked Ensemble Regression (MSER)

Mutual Information Feature Selection (MIFS) Based Crop Yield Prediction on Corn and Soybean Crops Using Multilayer Stacked Ensemble Regression (MSER)

Improved algorithm for matched-pairs selection of informative features in the problems of recognition of complex system states

The problem of computer diagnostics of complex systems is one of the non-trivial tasks of modern information technology. Such systems are, for example, computer networks, automatic and/or automated control systems for complex technological objects, including related to complex problems of environmental protection, biology, etc. In pattern recognition, one of the major problems is forming subspaces of informative features, which only in the «ensemble» allow diagnosing the states of such systems with a high degree of reliability. An effective approach to solving this problem based on the principles of inductive modeling of complex systems is proposed. The quality criterion for recognizing classes of patterns is formulated, which also makes it possible to evaluate the quality of the constructed ensemble of informative features. As an example, the problem of constructing an ensemble of informative features represented by a binary code based on the data of an experiment to determine the hazard levels of some plant protection products is considered. Real primary data on plant protection products used in practice were applied to recognize the effect of certain characteristics on the so-called integrated «hazard indicator». Comparative numerical estimates of the effectiveness of the proposed approach are given. In this case, there can be a fivefold gain in the amount of computations for a relatively small number of input features equal to 5 compared to the known algorithms of the class considered in the paper. It is shown that, from a practical point of view, the described algorithm has advantages over the known algorithms with brute-force search of feature subspaces in pattern recognition problems.

EEG Feature Selection via Global Redundancy Minimization for Emotion Recognition

A common drawback of EEG-based emotion recognition is that volume conduction effects of the human head introduce interchannel dependence and result in highly correlated information among most EEG features. These highly correlated EEG features cannot provide extra useful information, and they actually reduce the performance of emotion recognition. However, the existing feature selection methods, commonly used to remove redundant EEG features for emotion recognition, ignore the correlation between the EEG features or utilize a greedy strategy to evaluate the interdependence, which leads to the algorithms retaining the correlated and redundant features with similar feature scores in the EEG feature subset. To solve this problem, we propose a novel EEG feature selection method for emotion recognition, termed global redundancy minimization in orthogonal regression (GRMOR). GRMOR can effectively evaluate the dependence among all EEG features from a global view and then select a discriminative and nonredundant EEG feature subset for emotion recognition. To verify the performance of GRMOR, we utilized three EEG emotional data sets (DEAP, SEED, and HDED) with different numbers of channels (32, 62, and 128). The experimental results demonstrate that GRMOR is a promising tool for redundant feature removal and informative feature selection from highly correlated EEG features.

Efficient Approach of Automatic Speech Emotion Recognition (ASR) Using Mutual Information

Speech is a significant quality for distinguishing a person in daily human to human interaction/ communication. Like other biometric measures, such as face, iris and fingerprints, voice can therefore be used as a biometric measure for perceiving or identifying the person. Speaker recognition is almost the same as a kind of voice recognition in which the speaker is identified from the expression instead of the message. Automatic Speaker Recognition (ASR) is the way to identify people who rely on highlights that are omitted from speech expressions. Speech signals are awesome correspondence media that constantly pass on rich and useful knowledge, such as a speaker's feeling, sexual orientation, complement, and other interesting attributes. In any speaker identification, the essential task is to delete helpful highlights and allow for significant examples of speaker models.
 Hypothetical description, organization of the full state of feeling and the modalities of articulation of feeling are added. A SER framework is developed to conduct this investigation, in view of different classifiers and different techniques for extracting highlights. In this work various machine learning algorithms are investigated to identify decision boundary in feature space of audio signals. Moreover novelty of this art lies in improving the performance of classical machine learning algorithms using information theory based feature selection methods. The higher accuracy retrieved is 96 percent using Random forest algorithm incorporated with Joint Mutual information feature selection method.

Towards novel deep neuroevolution models: chaotic levy grasshopper optimization for short-term wind speed forecasting

High accurate wind speed forecasting plays an important role in ensuring the sustainability of wind power utilization. Although deep neural networks (DNNs) have been recently applied to wind time-series datasets, their maximum performance largely leans on their designed architecture. By the current state-of-the-art DNNs, their architectures are mainly configured in manual way, which is a time-consuming task. Thus, it is difficult and frustrating for regular users who do not have comprehensive experience in DNNs to design their optimal architectures to forecast problems of interest. This paper proposes a novel framework to optimize the hyperparameters and architecture of DNNs used for wind speed forecasting. Thus, we introduce a novel enhanced version of the grasshopper optimization algorithm called EGOA to optimize the deep long short-term memory (LSTM) neural network architecture, which optimally evolves four of its key hyperparameters. For designing the enhanced version of GOA, the chaotic theory and levy flight strategies are applied to make an efficient balance between the exploitation and exploration phases of the GOA. Moreover, the mutual information (MI) feature selection algorithm is utilized to select more correlated and effective historical wind speed time series features. The proposed model’s performance is comprehensively evaluated on two datasets gathered from the wind stations located in the United States (US) for two forecasting horizons of the next 30-min and 1-h ahead. The experimental results reveal that the proposed model achieves the best forecasting performance compared to seven prominent classical and state-of-the-art forecasting algorithms.