Normalized Mutual Information Feature Selection Research Articles

Feature Selection, Clustering and IoMT on Biomedical Engineering for COVID-19 Pandemic: A Comprehensive Review

In this era, feature clustering is a prominent technique in data mining. Features clustering have also huge application in biomedical research for multiple purpose including grouping, features reduction and many more. The Internet of Medical Things (IoMT) is a promising and emerging field of research that is having a major impact on knowledge retrieval and networking. IoMT also has significant application in biomedical research regarding remote monitoring and remote healthcare services. In this COVID-19 pandemic situation, psychological effect and human reaction have become a major concern of further research. A dataset can be reduced in size by using feature selection techniques. To facilitate subsequent processing, this will make the data easier to manage. Feature selection is also used to clean, reduce, and reduce dimensions of data. The clustering method has proven to be a powerful tool for finding patterns and structure in both labeled and unlabeled data sets. Our study basically provides various state-of-the-art methods regarding medical IoMT for remote healthcare, feature clustering for information retrieval regarding biomedical science. In this study, we are used five different type of feature selection like Minimum Redundancy - Maximum Relevance (mRMR), Random Forest, Normalized Mutual Information Feature Selection (NMIFS), F-Test and Chi-Square and five different type of Clustering algorithms like Hierarchical Clustering, Density-based spatial clustering of applications with noise (DBSCAN) Clustering, K-Means Clustering, Shrinkage Clustering, and Fuzzy C-Means Clustering. Finally, this study is very useful to understand and apply the appropriate IoMT, feature clustering, and catharsis on the various biomedical applications for the benevolence of society.

Development of a Novel Soft Sensor with Long Short-Term Memory Network and Normalized Mutual Information Feature Selection

In this paper, a novel soft sensor is developed by combining long short-term memory (LSTM) network with normalized mutual information feature selection (NMIFS). In the proposed algorithm, LSTM is designed to handle time series with high nonlinearity and dynamics of industrial processes. NMIFS is conducted to perform the input variable selection for LSTM to simplify the excessive complexity of the model. The developed soft sensor combines the excellent dynamic modelling of LSTM and precise variable selection of NMIFS. Simulations on two actual production datasets are used to demonstrate the performance of the proposed algorithm. The developed soft sensor could precisely predict the objective variables and has better performance than other methods.

An Improved Normalized Mutual Information Variable Selection Algorithm for Neural Network-Based Soft Sensors.

In this paper, normalized mutual information feature selection (NMIFS) and tabu search (TS) are integrated to develop a new variable selection algorithm for soft sensors. NMIFS is applied to select influential variables contributing to the output variable and avoids selecting redundant variables by calculating mutual information (MI). A TS based strategy is designed to prevent NMIFS from falling into a local optimal solution. The proposed algorithm performs the variable selection by combining the entropy information and MI and validating error information of artificial neural networks (ANNs); therefore, it has advantages over previous MI-based variable selection algorithms. Several simulation datasets with different scales, correlations and noise parameters are implemented to demonstrate the performance of the proposed algorithm. A set of actual production data from a power plant is also used to check the performance of these algorithms. The experiments showed that the developed variable selection algorithm presents better model accuracy with fewer selected variables, compared with other state-of-the-art methods. The application of this algorithm to soft sensors can achieve reliable results.

Various Feature Selection Techniques in Type 2 Diabetic Patients for the Prediction of Cardiovascular Disease

Cardiovascular disease (CVD) is a serious but preventable complication of type 2 diabetes mellitus (T2DM) that results in substantial disease burden, increased health services use, and higher risk of premature mortality [10]. People with diabetes are also at a greatly increased risk of cardiovascular which results in sudden death, which increases year by year. Data mining is the search for relationships and global patterns that exist in large databases but are `hidden' among the vast amount of data, such as a relationship between patient data and their medical diagnosis. Usually medical databases of type 2 diabetic patients are high dimensional in nature. If a training dataset contains irrelevant and redundant features (i.e., attributes), classification analysis may produce less accurate results. In order for data mining algorithms to perform efficiently and effectively on high-dimensional data, it is imperative to remove irrelevant and redundant features. Feature selection is one of the important and frequently used data preprocessing techniques for data mining applications in medicine. Many of the research area in data mining has improved the predictive accuracy of the classifiers by applying the various techniques of feature selection This paper illustrates, the application of feature selection technique in medical databases, will enable to find small number of informative features leading to potential improvement in medical diagnosis. It is proposed to find an optimal feature subset of the PIMA Indian Diabetes Dataset using Artificial Bee Colony technique with Differential Evolution, Symmetrical Uncertainty Attribute set Evaluator and Fast Correlation-Based Filter (FCBF). Then Mutual information based feature selection is done by introducing normalized mutual information feature selection (NMIFS). And valid classes of input features are selected by applying Hybrid Fuzzy C Means algorithm (HFCM).

An Anomaly Detection Method Based on Normalized Mutual Information Feature Selection and Quantum Wavelet Neural Network

This paper presents an anomaly detection model based on normalized mutual information feature selection (NMIFS) and quantum wavelet neural network (QWNN). The goal of the proposed model is to address the problem of determining the feature subset used to detect an anomaly in a machine learning task. In order to achieve an effective reduction for high-dimensional feature data, the NMIFS method is used to select the best feature combination from a given set of sample features. Then, the best combination of feature vectors are sent to the QWNN classifier for learning and training in the training phase, and the anomaly detection model is obtained. At the detection stage, the data is fed into the detection model and ultimately generates accurate detection results. The learning algorithm of structural risk minimization extreme learning machine is employed by the QWNN classifier to account for empirical and confidence risk. The experimental results on real abnormal data demonstrate that the NMIFS–QWNN method has higher detection accuracy and a lower false negative rate than the existing common anomaly detection methods. Furthermore, the complexity of the algorithm is low and the detection accuracy can reach up to 95.8%.

Heterogeneous feature subset selection using mutual information-based feature transformation

Conventional mutual information (MI) based feature selection (FS) methods are unable to handle heterogeneous feature subset selection properly because of data format differences or estimation methods of MI between feature subset and class label. A way to solve this problem is feature transformation (FT). In this study, a novel unsupervised feature transformation (UFT) which can transform non-numerical features into numerical features is developed and tested. The UFT process is MI-based and independent of class label. MI-based FS algorithms, such as Parzen window feature selector (PWFS), minimum redundancy maximum relevance feature selection (mRMR), and normalized MI feature selection (NMIFS), can all adopt UFT for pre-processing of non-numerical features. Unlike traditional FT methods, the proposed UFT is unbiased while PWFS is utilized to its full advantage. Simulations and analyses of large-scale datasets showed that feature subset selected by the integrated method, UFT–PWFS, outperformed other FT–FS integrated methods in classification accuracy.

Mass Classification in Mammograms Using Selected Geometry and Texture Features, and a New SVM-Based Feature Selection Method

Masses are the primary indications of breast cancer in mammograms, and it is important to classify them as benign or malignant. Benign and malignant masses differ in geometry and texture characteristics. However, not every geometry and texture feature that is extracted contributes to the improvement of classification accuracy; thus, to select the best features from a set is important. In this paper, we examine the feature selection methods for mass classification. We integrate a support vector machine (SVM)-based recursive feature elimination (SVM-RFE) procedure with a normalized mutual information feature selection (NMIFS) to avoid their singular disadvantages (the redundancy in the selected features of the SVM-RFE and the unoptimized classifier for the NMIFS) while retaining their advantages, and we propose a new feature selection method, which is called the SVM-RFE with an NMIFS filter (SRN). In addition to feature selection, we also study the initialization of mass segmentation. Different initialization methods are investigated, and we propose a fuzzy c-means (FCM) clustering, with spatial constraints as the initialization step. In the experiments, 826 regions of interest (ROIs) from the Digital Database for Screening Mammography were used. All 826 were used in the classification experiments, and 413 ROIs were used in the feature selection experiments. Different feature selection methods, including F-score, Relief, SVM-RFE, SVM-RFE with a minimum redundancy-maximum relevance (mRMR) filter [SVM-RFE (mRMR)], and SRN, were used to select features and to compare mass classification results using the selected features. In the classification experiments, the linear discriminant analysis and the SVM classifiers were investigated. The accuracy that is obtained with the SVM classifier using the selected features obtained by the F-score, Relief, SVM-RFE, SVM-RFE (mRMR), and SRN methods are 88%, 88%, 90%, 91%, and 93%, respectively, with a tenfold cross-validation procedure, and 91%, 89%, 92%, 92%, and 94%, respectively, with a leave-one-out (LOO) scheme. We also compared the performance of the different feature selection methods using the receiver operating characteristic analysis and the areas under the curve (AUCs). The AUCs for the F-score, Relief, SVM-RFE, SVM-RFE (mRMR), and SRN methods are 0.9014, 0.8916, 0.9121, 0.9236, and 0.9439, respectively, with a tenfold cross-validation procedure, and are 0.9312, 0.9178, 0.9324, 0.9413, and 0.9615, respectively, with a LOO scheme. Both the accuracy and AUC values show that the proposed SRN feature selection method has the best performance. In addition to the accuracy and the AUC, we also measured the significance between the two best feature selection methods, i.e., the SVM-RFE (mRMR) and the proposed SRN method. Experimental results show that the proposed SRN method is significantly more accurate than the SVM-RFE (mRMR) (p = 0.011).

Gender Classification Based on Fusion of Different Spatial Scale Features Selected by Mutual Information From Histogram of LBP, Intensity, and Shape

In this paper, we report our extension of the use of feature selection based on mutual information and feature fusion to improve gender classification of face images. We compare the results of fusing three groups of features, three spatial scales, and four different mutual information measures to select features. We also showed improved results by fusion of LBP features with different radii and spatial scales, and the selection of features using mutual information. As measures of mutual information we use minimum redundancy and maximal relevance (mRMR), normalized mutual information feature selection (NMIFS), conditional mutual information feature selection (CMIFS), and conditional mutual information maximization (CMIM). We tested the results on four databases: FERET and UND, under controlled conditions, the LFW database under unconstrained scenarios, and AR for occlusions. It is shown that selection of features together with fusion of LBP features significantly improved gender classification accuracy compared to previously published results. We also show a significant reduction in processing time because of the feature selection, which makes real-time applications of gender classification feasible.

Gender Classification From Face Images Using Mutual Information and Feature Fusion

In this article we report a new method for gender classification from frontal face images using feature selection based on mutual information and fusion of features extracted from intensity, shape, texture, and from three different spatial scales. We compare the results of three different mutual information measures: minimum redundancy and maximal relevance (mRMR), normalized mutual information feature selection (NMIFS), and conditional mutual information feature selection (CMIFS). We also show that by fusing features extracted from six different methods we significantly improve the gender classification results relative to those previously published, yielding 99.13% of the gender classification rate on the FERET database.

Normalized Mutual Information Feature Selection

A filter method of feature selection based on mutual information, called normalized mutual information feature selection (NMIFS), is presented. NMIFS is an enhancement over Battiti's MIFS, MIFS-U, and mRMR methods. The average normalized mutual information is proposed as a measure of redundancy among features. NMIFS outperformed MIFS, MIFS-U, and mRMR on several artificial and benchmark data sets without requiring a user-defined parameter. In addition, NMIFS is combined with a genetic algorithm to form a hybrid filter/wrapper method called GAMIFS. This includes an initialization procedure and a mutation operator based on NMIFS to speed up the convergence of the genetic algorithm. GAMIFS overcomes the limitations of incremental search algorithms that are unable to find dependencies between groups of features.