Mutual Information Feature Selection Research Articles

An Intrusion Detection System Based on Normalized Mutual Information Antibodies Feature Selection and Adaptive Quantum Artificial Immune System

The intrusion detection system (IDS) has lower speed, less adaptability and lower detection accuracy especially for small samples sets. This paper presents a detection model based on normalized mutual antibodies information feature selection and adaptive quantum artificial immune with cooperative evolution of multiple operators (NMAIFS MOP-AQAI). First, for a high intrusion speed, the NMAIFS is used to achieve an effective reduction for high-dimensional features. Then, the best feature vectors are sent to the MOP-AQAI classifier, in which, vaccination strategy, the quantum computing, and cooperative evolution of multiple operators are adopted to generate excellent detectors. Lastly, the data is fed into NMAIFS MOP-AQAI and ultimately generates accurate detection results. The experimental results on real abnormal data demonstrate that the NMAIFS MOP-AQAI has higher detection accuracy, lower false negative rate and a higher adaptive performance than the existing anomaly detection methods, especially for small samples sets.

Intrusion Detection Using Normalized Mutual Information Feature Selection and Parallel Quantum Genetic Algorithm

This paper presents a detection algorithm using normalized mutual information feature selection and cooperative evolution of multiple operators based on adaptive parallel quantum genetic algorithm (NMIFS MOP- AQGA). The proposed algorithm is to address the problems that the intrusion detection system (IDS) has lower the detection speed, less adaptability and lower detection accuracy. In order to achieve an effective reduction for high-dimensional feature data, the NMIFS method is used to select the best feature combination. The best features are sent to the MOP- AQGA classifier for learning and training, and the intrusion detectors are obtained. The data are fed into the detection algorithm to ultimately generate accurate detection results. The experimental results on real abnormal data demonstrate that the NMIFS MOP- AQGA method has higher detection accuracy, lower false negative rate and higher adaptive performance than the existing detection methods, especially for small samples sets.

Diagnosis of Parkinson's disease based on SHAP value feature selection

To address the problem of high feature dimensionality of Parkinson's disease medical data, this paper introduces SHapley Additive exPlanations (SHAP) value for feature selection of Parkinson's disease medical dataset. This paper combines SHAP value with four classifiers, namely deep forest (gcForest), extreme gradient boosting (XGBoost), light gradient boosting machine (LightGBM) and random forest (RF), respectively. Then this paper applies them to Parkinson's disease diagnosis. First, the classifier is used to calculate the magnitude of contribution of SHAP value to the features, then the features with significant contribution in the classification task are selected, and then the data after feature selection is used as input to classify the Parkinson's disease dataset for diagnosis using the classifier. The experimental results show that compared to Fscore, analysis of variance (Anova-F) and mutual information (MI) feature selection methods, the four models based on SHAP-value feature selection achieved good classification results. The SHAP-gcForest model combined with gcForest achieves classification accuracy of 91.78% and F1-score of 0.945 when 150 features are selected. The SHAP-LightGBM model combined with LightGBM achieves classification accuracy and F1-score of 91.62% and 0.945 when 50 features are selected, respectively. The classification effectiveness is second only to the SHAP-gcForest model, but the SHAP-LightGBM model is more computationally efficient than the SHAP-gcForest model. Finally, the effectiveness of the proposed method is verified by comparing it with the results of existing literature. The findings demonstrate that machine learning with SHAP value feature selection method has good classification performance in the diagnosis of Parkinson's disease, and provides a reference for physicians in the diagnosis and prevention of Parkinson's disease.

Feature Engineering for Interpretable Machine Learning for Quality Assurance in Radiation Oncology.

Chart checking is a time intensive process with high cognitive workload for physicists. Previous studies have partially automated and standardized chart checking, but limited studies implement data-driven approaches to reduce cognitive workload for quality assurance processes. This study aims to evaluate feature selection methods to improve the interpretability and transparency of machine learning models in predicting the degree of difficulty for a pretreatment physics chart check. We compare chi-square, mutual information, feature importance thresholding, and greedy feature selection for four different classifiers. Random forest has the highest performance with SMOTE oversampling using mutual information for feature selection (accuracy 84.0%, AUC 87.0%, precision 80.0%, recall 80.0%). This study demonstrates that feature selection methods can improve model interpretability and transparency.

Prediction of pathological complete response to neoadjuvant chemotherapy in breast cancer using radiomics of pretreatment dynamic contrast-enhanced MRI

PurposeTo investigate if the pretreatment dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI)-based radiomics machine learning predicts the pathological complete response (pCR) to neoadjuvant chemotherapy (NAC) in breast cancer patients. MethodsSeventy-eight breast cancer patients who underwent DCE-MRI before NAC and confirmed as pCR or non-pCR were enrolled. Early enhancement mapping images of pretreatment DCE-MRI were created using subtraction formula as follows: Early enhancement mapping = (Signal 1 min – Signal pre)/Signal pre. Images of the whole tumors were manually segmented and radiomics features extracted. Five prediction models were built using five scenarios that included clinical information, subjective radiological findings, first order texture features, second order texture features, and their combinations. In texture analysis workflow, the corresponding variables were identified by mutual information for feature selection and random forest was used for model prediction. In five models, the area under the receiver operating characteristic curves (AUC) to predict the pCR and several metrics for model evaluation were analyzed. ResultsThe best diagnostic performance based on F-score was achieved when both first and second order texture features with clinical information and subjective radiological findings were used (AUC = 0.77). The second best diagnostic performance was achieved with an AUC of 0.76 for first order texture features followed by an AUC of 0.76 for first and second order texture features. ConclusionsPretreatment DCE-MRI can improve the prediction of pCR in breast cancer patients when all texture features with clinical information and subjective radiological findings are input to build the prediction model.

Prediction of NOx Concentration at SCR Inlet Based on BMIFS-LSTM

As the main energy source for thermal power generation, coal generates a large amount of NOx during its incineration in boilers, and excessive NOx emissions can cause serious pollution to the air environment. Selective catalytic reduction denitrification (SCR) selects the optimal amount of ammonia to be injected for denitrification based on the measurement of NOx concentration by the automatic flue gas monitoring system. Since the automatic flue gas monitoring system has a large delay in measurement, it cannot accurately reflect the real-time changes of NOx concentration at the SCR inlet when the unit load fluctuates, leading to problems such as ammonia escape and NOx emission exceeding the standard. In response to these problems, this paper proposes an SCR inlet NOx concentration prediction algorithm based on BMIFS-LSTM. An improved mutual information feature selection algorithm (BMIFS) is used to filter out the auxiliary variables with maximum correlation and minimum redundancy with NOx concentration, and reduce the coupling and dimensionality among the variables in the data set. The dominant and auxiliary variables are then fed together into a long short-term memory neural network (LSTM) to build a prognostic model. Simulation experiments are conducted using historical operation data of a 300 MW thermal power unit. The experimental results show that the algorithm in this paper reduces the average relative error by 3.45% and the root mean square error by 1.50 compared with the algorithm without auxiliary variable extraction, which can accurately reflect the real-time changes of NOx concentration at the SCR inlet, solve the problem of delay in NOx concentration measurement, and reduce the occurrence of atmospheric pollution caused by excessive NOx emissions.

An Unsupervised Mutual Information Feature Selection Method Based on SVM for Main Transformer Condition Diagnosis in Nuclear Power Plants

Dissolved gas in oil (DGA) is a common means of monitoring the condition of an oil-immersed transformer. The concentration of dissolved gas and the ratio of different gases are important indexes to judge the condition of power transformers. Monitoring devices for dissolved gas in oil are widely installed in main transformers, but there are few recorded fault data of main transformers. The special operation and maintenance modes of main transformers leads to the fault modes particularity of main transformers. In order to solve the problem of insufficient samples and the feature uncertainty, this paper puts forward an unsupervised mutual information method to select the feature verified by the optimized support vector machine (SVM) model of particle swarm optimization (PSO) method and tries to find the feature sequence with better performance. The methos is validated by data from nuclear power transformers.

Application of multidomain sensor image fusion and training data augmentation for enhanced CNN image classification

Convolutional neural networks (CNNs) provide the sensing and detection community with a discriminative approach for classifying images. However, one of the largest limitations of deep CNN image classifiers is the need for extensive training datasets containing a variety of image representations. While current methods, such as generative adversarial network data augmentation, additions of noise, rotations, and translations, can allow CNNs to better associate new images and their feature representations to ones of a learned image class, many fail to provide new contexts of ground truth feature information. To expand the association of critical class features within CNN image training datasets, an image pairing and training dataset augmentation paradigm via a multi-sensor domain image data fusion algorithm is proposed. This algorithm uses a mutual information (MI) and merit-based feature selection subroutine to pair highly correlated cross-domain images from multiple sensor domain image datasets. It then re-augments the corresponding cross-domain image pairs into the opposite sensor domain’s feature set via a highest MI, cross sensor domain, and image concatenation function. This augmented image set then acts to retrain the CNN to recognize greater generalizations of image class features via cross domain, mixed representations. Experimental results indicated an increased ability of CNNs to generalize and discriminate between image classes during testing of class images from synthetic aperture radar vehicle, solar cell device reliability screening, and lung cancer detection image datasets.

Fuzzy joint mutual information feature selection based on ideal vector

Feature selection is an effective preprocess that helps the classification task to improve its performance. Information measures, especially fuzzy information measures have been a powerful solution in feature selection. However, existing feature selection frameworks based on fuzzy information measures suffer from one of two limitations: (1) discarding important feature relations as redundancy and complementarity. (2) requiring a high computational cost. To overcome these limitations, a new feature selection framework is proposed in this paper, called fuzzy joint mutual information based on ideal vector (FJMIIV), to extract all possible feature relations with minimum computational cost. The performance of the proposed framework is compared with some of the state-of-the-art frameworks on various benchmark datasets. The comparison results using four well-known classifiers show that the proposed method can effectively improve the classification performance.

Computer‐aided diagnosis ofCOVID‐19 disease from chest X‐ray images integrating deep feature extraction

AbstractThe novel coronavirus (COVID‐19) has an enormous impact on the daily lives and health of people residing in more than 200 nations. This article proposes a deep learning‐based system for the rapid diagnosis of COVID‐19. Chest x‐ray radiograph images were used because recent findings revealed that these images contain salient features about COVID‐19 disease. Transfer learning was performed using different pre‐trained convolutional neural networks models for binary (normal and COVID‐19) and triple (normal, COVID‐19 and viral pneumonia) class problems. Deep features were extracted from a fully connected layer of the ResNET50v2 model and feature dimension was reduced through feature reduction methods. Feature fusion of feature sets reduced through analysis of variance (ANOVA) and mutual information feature selection (MIFS) was fed to Fine K‐nearest neighbour to perform binary classification. Similarly, serial feature fusion of MIFS and chi‐square features were utilized to train Medium Gaussian Support Vector Machines to distinguish normal, COVID‐19 and viral pneumonia cases. The proposed framework yielded accuracies of 99.5% for binary and 95.5% for triple class experiments. The proposed model shows better performance than the existing methods, and this research has the potential to assist medical professionals to enhance the diagnostic ability to detect coronavirus disease.

Analyzing the Features Affecting the Performance of Teachers during Covid-19: A Multilevel Feature Selection

COVID-19 is a profoundly contagious pandemic that has taken the world by storm and has afflicted different fields of life with negative effects. It has had a substantial impact on education which is evident from the transition from Face-to-Face (F2F) teaching to online mode of education and the rigid implementation of lockdown across the globe. This study examines the factors impacting the performance of teachers during the lockdown period of COVID-19 using various feature selection algorithms and Artificial Intelligence techniques. In this paper, we have proposed a novel multilevel feature selection for the prediction of the factors affecting the teachers’ satisfaction with online teaching and learning in COVID-19. The proposed multilevel feature selection is composed of the Fast Correlation Based Filter (FCBF), Mutual Information (MI), Relieff, and Particle Swarm Optimization (PSO) feature selection. The performance of the proposed feature selection approach is evaluated through the teachers’ benchmark dataset. We used a range of measures like accuracy, precision, f-measure, and recall to evaluate the performance of the proposed approach. We applied different machine learning approaches (SVM, LGBM, and ANN) with the proposed multilevel feature selection technique. The performance of the proposed approach is also compared with existing feature selection algorithms, and the results show the improvement in the performance of feature selection in terms of accuracy, precision, recall, and F-Measure. Proposed feature selection provides more than 80% accuracy with Light Weight Gradient Boosting Machine (LGBM).

Probabilistic decision support system using machine learning techniques : A case study of Cardiovascular diseases

Diseases are an irregular event that affects single or multiple body parts of a human being. Different kinds of diseases are growing day by day in terms of lifestyle and heritage. Among all the ones, it turns out that heart disease is the most prevalent disease, and the consequence of this disease is dangerous relative to all other situations. In this paper, initially the author calculates immanent discreteness between two probability separations, based on information theory or relative entropy using Kullback-Leibler divergence (KL divergence). It can be used as shortcuts in the calculation of other widely used methods, such as mutual information for feature selection before modeling, and cross-entropy used as a loss function for many different classifier models. Seven computational intelligence techniques named logistic regression (LR), support vector machine (SVM), naïve bayes (NB), random forest (RF), and k-nearest neighbor (K-NN) have been applied and a comparative study had been drawn. The heart disease dataset is taken from the Department of Cardiology, Excelcare Hospitals, Guwahati, Assam and the analysis of performance in each technique has been evaluated.

Mutual Information Feature Selection (MIFS) Based Crop Yield Prediction on Corn and Soybean Crops Using Multilayer Stacked Ensemble Regression (MSER)

Mutual Information Feature Selection (MIFS) Based Crop Yield Prediction on Corn and Soybean Crops Using Multilayer Stacked Ensemble Regression (MSER)

Efficient Approach of Automatic Speech Emotion Recognition (ASR) Using Mutual Information

Speech is a significant quality for distinguishing a person in daily human to human interaction/ communication. Like other biometric measures, such as face, iris and fingerprints, voice can therefore be used as a biometric measure for perceiving or identifying the person. Speaker recognition is almost the same as a kind of voice recognition in which the speaker is identified from the expression instead of the message. Automatic Speaker Recognition (ASR) is the way to identify people who rely on highlights that are omitted from speech expressions. Speech signals are awesome correspondence media that constantly pass on rich and useful knowledge, such as a speaker's feeling, sexual orientation, complement, and other interesting attributes. In any speaker identification, the essential task is to delete helpful highlights and allow for significant examples of speaker models.
 Hypothetical description, organization of the full state of feeling and the modalities of articulation of feeling are added. A SER framework is developed to conduct this investigation, in view of different classifiers and different techniques for extracting highlights. In this work various machine learning algorithms are investigated to identify decision boundary in feature space of audio signals. Moreover novelty of this art lies in improving the performance of classical machine learning algorithms using information theory based feature selection methods. The higher accuracy retrieved is 96 percent using Random forest algorithm incorporated with Joint Mutual information feature selection method.

Towards novel deep neuroevolution models: chaotic levy grasshopper optimization for short-term wind speed forecasting

High accurate wind speed forecasting plays an important role in ensuring the sustainability of wind power utilization. Although deep neural networks (DNNs) have been recently applied to wind time-series datasets, their maximum performance largely leans on their designed architecture. By the current state-of-the-art DNNs, their architectures are mainly configured in manual way, which is a time-consuming task. Thus, it is difficult and frustrating for regular users who do not have comprehensive experience in DNNs to design their optimal architectures to forecast problems of interest. This paper proposes a novel framework to optimize the hyperparameters and architecture of DNNs used for wind speed forecasting. Thus, we introduce a novel enhanced version of the grasshopper optimization algorithm called EGOA to optimize the deep long short-term memory (LSTM) neural network architecture, which optimally evolves four of its key hyperparameters. For designing the enhanced version of GOA, the chaotic theory and levy flight strategies are applied to make an efficient balance between the exploitation and exploration phases of the GOA. Moreover, the mutual information (MI) feature selection algorithm is utilized to select more correlated and effective historical wind speed time series features. The proposed model’s performance is comprehensively evaluated on two datasets gathered from the wind stations located in the United States (US) for two forecasting horizons of the next 30-min and 1-h ahead. The experimental results reveal that the proposed model achieves the best forecasting performance compared to seven prominent classical and state-of-the-art forecasting algorithms.

An Improvised Machine Learning Model Based on Mutual Information Feature Selection Approach for Microbes Classification.

The accurate classification of microbes is critical in today’s context for monitoring the ecological balance of a habitat. Hence, in this research work, a novel method to automate the process of identifying microorganisms has been implemented. To extract the bodies of microorganisms accurately, a generalized segmentation mechanism which consists of a combination of convolution filter (Kirsch) and a variance-based pixel clustering algorithm (Otsu) is proposed. With exhaustive corroboration, a set of twenty-five features were identified to map the characteristics and morphology for all kinds of microbes. Multiple techniques for feature selection were tested and it was found that mutual information (MI)-based models gave the best performance. Exhaustive hyperparameter tuning of multilayer layer perceptron (MLP), k-nearest neighbors (KNN), quadratic discriminant analysis (QDA), logistic regression (LR), and support vector machine (SVM) was done. It was found that SVM radial required further improvisation to attain a maximum possible level of accuracy. Comparative analysis between SVM and improvised SVM (ISVM) through a 10-fold cross validation method ultimately showed that ISVM resulted in a 2% higher performance in terms of accuracy (98.2%), precision (98.2%), recall (98.1%), and F1 score (98.1%).

Rank-based univariate feature selection methods on machine learning classifiers for code smell detection

Detecting code smells and treating them with refactoring are trivial part of maintaining vast and sophisticated software. There is an urgent need for automatic system to treat code smells. Tools provide variable results, based on threshold values and subjective interpretation of smells. Machine learning is one of the best approaches that provides effective solution to this problem. Practitioners do not need expert knowledge on smell’s characteristics for detection, which makes this approach accessible. In this paper, we have implemented 32 machine learning algorithms after performing feature selection through six variations of the filter method. We have used multiple correlation methodologies to discard similar features. Mutual information, fisher score, and univariate ROC–AUC feature selection techniques were used with brute force and random forest correlation strategies. Feature selection eliminates dimensionality curse and improves performance measures drastically. It is the selection of relevant feature subset based on the relation between dependent and independent variables. We have compared performance of classifiers implemented with and without performing feature selection. Results show that accuracy of machine learning models has increased up to 26.5%, f-measure by 70.9%, area under ROC curve has surged up to 26.74%, and average training time has reduced up to 62 s as compared to performance measures of machine learning models executed without feature selection. Mutual information feature selection strategy with random forest correlation methodology has the highest impact on performance measures among all the filter methods. Among 32 classifiers, boosted decision trees (J48) and Naive Bayes algorithms gave best performance after dimensionality reduction.

Project cost Management Strategy based on Big Data and BIM

With the development of socialist market economy, the current project cost management system in China has gradually exposed some disadvantages, which affect the normal operation and development of construction enterprises to a certain extent. This paper analyzes the research status of PCM, discusses the characteristics of BD technology, the concept of BIMT and the concept of PCM, and finds the method of mutual information feature selection. The results show that in 2020, the output value of construction projects in China is the largest, with 5.54 billion yuan in the first quarter, 5.83 billion yuan in the second quarter, and 6.77 billion yuan in the third quarter. Due to the large number of participants, long construction period and large amount of information in the construction process, these greatly increase the difficulty of PM, but also increase the difficulty of PCM, making some cost problems easier to appear.

Research on Spam Filtering Technology Based on New Mutual Information Feature Selection Algorithm

Aiming at the deficiency of traditional mutual information algorithm in feature selection, this paper proposes a weighted naive Bayesian algorithm based on improved mutual information, called imi-wnb algorithm. In the feature selection stage, the word frequency factor and the difference factor between classes are introduced to improve the traditional mutual information algorithm to achieve feature dimension reduction. In the process of classification, the value of IMI is introduced to weight the attributes of naive Bayes algorithm, which partly eliminates the influence of conditional independence assumption of naive Bayes algorithm on classification, and improves the efficiency and stability of spam classification.

Machine learning application identifies novel gene signatures from transcriptomic data of spontaneous canine hemangiosarcoma.

Angiosarcomas are soft-tissue sarcomas that form malignant vascular tissues. Angiosarcomas are very rare, and due to their aggressive behavior and high metastatic propensity, they have poor clinical outcomes. Hemangiosarcomas commonly occur in domestic dogs, and share pathological and clinical features with human angiosarcomas. Typical pathognomonic features of this tumor are irregular vascular channels that are filled with blood and are lined by a mixture of malignant and nonmalignant endothelial cells. The current gold standard is the histological diagnosis of angiosarcoma; however, microscopic evaluation may be complicated, particularly when tumor cells are undetectable due to the presence of excessive amounts of nontumor cells or when tissue specimens have insufficient tumor content. In this study, we implemented machine learning applications from next-generation transcriptomic data of canine hemangiosarcoma tumor samples (n = 76) and nonmalignant tissues (n = 10) to evaluate their training performance for diagnostic utility. The 10-fold cross-validation test and multiple feature selection methods were applied. We found that extra trees and random forest learning models were the best classifiers for hemangiosarcoma in our testing datasets. We also identified novel gene signatures using the mutual information and Monte Carlo feature selection method. The extra trees model revealed high classification accuracy for hemangiosarcoma in validation sets. We demonstrate that high-throughput sequencing data of canine hemangiosarcoma are trainable for machine learning applications. Furthermore, our approach enables us to identify novel gene signatures as reliable determinants of hemangiosarcoma, providing significant insights into the development of potential applications for this vascular malignancy.