Supervised Feature Selection Technique Research Articles

Explainable machine learning models for Medicare fraud detection

As a means of building explainable machine learning models for Big Data, we apply a novel ensemble supervised feature selection technique. The technique is applied to publicly available insurance claims data from the United States public health insurance program, Medicare. We approach Medicare insurance fraud detection as a supervised machine learning task of anomaly detection through the classification of highly imbalanced Big Data. Our objectives for feature selection are to increase efficiency in model training, and to develop more explainable machine learning models for fraud detection. Using two Big Data datasets derived from two different sources of insurance claims data, we demonstrate how our feature selection technique reduces the dimensionality of the datasets by approximately 87.5% without compromising performance. Moreover, the reduction in dimensionality results in machine learning models that are easier to explain, and less prone to overfitting. Therefore, our primary contribution of the exposition of our novel feature selection technique leads to a further contribution to the application domain of automated Medicare insurance fraud detection. We utilize our feature selection technique to provide an explanation of our fraud detection models in terms of the definitions of the selected features. The ensemble supervised feature selection technique we present is flexible in that any collection of machine learning algorithms that maintain a list of feature importance values may be used. Therefore, researchers may easily employ variations of the technique we present.

Identifying Acoustic Features to Distinguish Highly and Moderately Altered Soundscapes in Colombia

Numerous acoustic features have been proposed as useful measures to characterize natural soundscapes, which can be employed to examine the impact of land transformation on the audible properties of a location. The extensive collection of available features demands an examination to identify the most informative and discriminative ones for a given problem. In this study, we conduct an empirical investigation into the selection of acoustic features for discriminating between highly and moderately transformed versions of four Colombian soundscapes: Moorlands, coffee plantations, dry tropical forests, and pastures. We employ classical supervised feature selection techniques along with exploratory tools such as correlation matrices and scatter plots. Our results indicate that a few acoustic features are sufficient to differentiate between the classes. Specifically, those features that estimate acoustic complexity via intrinsic variability of sound intensities or biodiversity through species richness or abundance in specific frequency bands are the most discriminative ones. These findings suggest that the selection of acoustic features can assist in analyzing and distinguishing between different soundscapes.

Designing a supervised feature selection technique for mixed attribute data analysis

Identifying optimal features is critical for increasing the overall performance of data classification. This paper introduces a supervised feature selection technique for analyzing mixed attribute data. It measures data classification performances of features with a user-defined performance criterion and determines optimal features to boost the overall data analysis performance. A performance evaluation is managed to highlight the usefulness of the technique with existing feature selection techniques such as analysis of variance test, chi-square test, principal component analysis, and mutual information. Visualization is also utilized to understand the differences in classifying instances with different features. From a comparative performance testing and evaluation, we found 5 ∼ 10% performance improvements with the proposed technique. Overall, evaluation results showed the usefulness of our proposed feature selection technique in mixed attribute data analysis.

Machine Learning Algorithms in Corroboration with Isotope and Elemental Profile—An Efficient Tool for Honey Geographical Origin Assessment

The application of artificial intelligence for the development of recognition models for food and beverages differentiation has benefited from increasing attention in recent years. For this scope, different machine learning (ML) algorithms were used in order to find the most suitable model for a certain purpose. In the present work, three ML algorithms, namely artificial neural networks (ANN), support vector machines (SVM) and k-nearest neighbors (KNN), were applied for constructing honey geographical classification models, and their performance was assessed and compared. A preprocessing step consisting of either a component reduction method or a supervised feature selection technique was applied prior to model development. The most efficient geographical differentiation models were obtained based on ANN, when a subset of features corresponding to the markers having the highest discrimination potential was used as input data. Therefore, when the samples aimed to be classified at an intercountry level, an accuracy of 95% was achieved; namely, 99% of the Romanian samples and 73% of the ones originating from other countries were correctly predicted. Promising results were also obtained for the intracountry honey discrimination; namely, the model built for classifying the Transylvanian samples from the ones produced in other Romanian regions had an 85% accuracy.

Gene Selection in Binary Classification Problems Within Functional Genomics Experiments via Robust Fisher Score

This study proposes a supervised feature selection technique for classification in high dimensional binary class problems by adding robustness in the conventional Fisher Score. The proposed method utilizes the more robust measure of location, i.e. the median and measure of dispersion known as Rousseeuw and Croux statistic (Qn). Initially, a minimum subset of genes is identified by the greedy search approach, which is then combined with the top ranked genes obtained via the proposed Robust Fisher Score (RFish). To remove redundancy in the selected genes, Least Absolute Shrinkage and Selection Operator (LASSO) is then applied. The proposed method is validated on five publicly available datasets and is further assessed in a detailed simulation study. The results of the proposed method are compared with six well known feature selection methods based on prediction performance via Random Forest (RF), Support Vector Machine (SVM) and k Nearest Neighbour (k-NN) classifiers. The findings are presented in boxplots and barplots, which show that the proposed method (RFish) outperforms all the other methods in the majority of cases.

Supervised feature selection techniques in network intrusion detection: A critical review

Machine Learning (ML) techniques are becoming an invaluable support for network intrusion detection, especially in revealing anomalous flows, which often hide cyber-threats. Typically, ML algorithms are exploited to classify/recognize data traffic on the basis of statistical features such as inter-arrival times, packets length distribution, mean number of flows, etc. Dealing with the vast diversity and number of features that typically characterize data traffic is a hard problem. This results in the following issues: (i) the presence of so many features leads to lengthy training processes (particularly when features are highly correlated), while prediction accuracy does not proportionally improve; (ii) some of the features may introduce bias during the classification process, particularly those that have scarce relation with the data traffic to be classified. To this end, by reducing the feature space and retaining only the most significant features, Feature Selection (FS) becomes a crucial pre-processing step in network management and, specifically, for the purposes of network intrusion detection. In this review paper, we complement other surveys in multiple ways: (i) evaluating more recent datasets (updated w.r.t. obsolete KDD 99) by means of a designed-from-scratch Python-based procedure; (ii) providing a synopsis of most credited FS approaches in the field of intrusion detection, including Multi-Objective Evolutionary techniques; (iii) assessing various experimental analyses such as feature correlation, time complexity, and performance. Our comparisons offer useful guidelines to network/security managers who are considering the incorporation of ML concepts into network intrusion detection, where trade-offs between performance and resource consumption are crucial.

A rough-GA based optimal feature selection in attribute profiles for classification of hyperspectral imagery

Morphological attribute profiles are robust in capturing the spectral–spatial information of hyperspectral imagery. To incorporate maximum spatial information, generation of a profile using multiple attributes with large number of threshold values is a well-known approach. Although the profile contains very rich spatial information, at the same time its dimensionality increases. This raises two critical problems for hyperspectral image classification: (i) curse of dimensionality and (ii) computational complexity. To mitigate such problems, the only supervised feature selection technique that exists in the literature is computationally demanding. In this article, a fast supervised feature selection technique by exploiting rough set theory and genetic algorithms is proposed. Our technique computes the relevance and significance of each feature in the profile using rough set theory. Then, based on the relevance and significance values a novel fitness function of genetic algorithms is designed to select an optimal subset of features from the constructed profile. To show the effectiveness of the proposed technique, it is compared with the existing state-of-the-art technique by considering three real hyperspectral data sets.

Semantic textual similarity between sentences using bilingual word semantics

Semantic textual similarity between sentences is indispensable for many information retrieval tasks. Traditional lexical similarity measures cannot compute the similarity beyond a trivial level. Moreover, they only can capture the textual similarity, but not semantic. In this paper, we propose a method for semantic textual similarity that leverages bilingual word-level semantics to compute the semantic similarity between sentences. To capture word-level semantics, we employ distribute representation of words in two different languages. The similarity function based on the concept-to-concept relationship corresponding to the words is also utilized for the same purpose. Multiple new semantic similarity measures are introduced based on word-embedding models trained on two different corpora in two different languages. Apart from these, another new semantic similarity measure is also introduced using the word sense comparison. The similarity score between the sentences is then computed by applying a linear ranking approach to all proposed measures with their importance score estimated employing a supervised feature selection technique. We conducted experiments on the SemEval Semantic Textual Similarity (STS-2017) test collections. The experimental results demonstrated that our method is effective for measuring semantic textual similarity and outperforms some known related methods.

Unsupervised Feature Selection Based on Ultrametricity and Sparse Training Data: A Case Study for the Classification of High-Dimensional Hyperspectral Data

In this paper, we investigate the potential of unsupervised feature selection techniques for classification tasks, where only sparse training data are available. This is motivated by the fact that unsupervised feature selection techniques combine the advantages of standard dimensionality reduction techniques (which only rely on the given feature vectors and not on the corresponding labels) and supervised feature selection techniques (which retain a subset of the original set of features). Thus, feature selection becomes independent of the given classification task and, consequently, a subset of generally versatile features is retained. We present different techniques relying on the topology of the given sparse training data. Thereby, the topology is described with an ultrametricity index. For the latter, we take into account the Murtagh Ultrametricity Index (MUI) which is defined on the basis of triangles within the given data and the Topological Ultrametricity Index (TUI) which is defined on the basis of a specific graph structure. In a case study addressing the classification of high-dimensional hyperspectral data based on sparse training data, we demonstrate the performance of the proposed unsupervised feature selection techniques in comparison to standard dimensionality reduction and supervised feature selection techniques on four commonly used benchmark datasets. The achieved classification results reveal that involving supervised feature selection techniques leads to similar classification results as involving unsupervised feature selection techniques, while the latter perform feature selection independently from the given classification task and thus deliver generally versatile features.

Analysis of Supervised Feature Selection Techniques on Animal Husbandry Dataset

Analysis of Supervised Feature Selection Techniques on Animal Husbandry Dataset

A hybrid isotonic separation training algorithm with correlation-based isotonic feature selection for binary classification

Isotonic separation is a classification technique which constructs a model by transforming the training set into a linear programming problem (LPP). It is computationally expensive to solve large-scale LPPs using traditional methods when data set grows. This paper proposes a hybrid binary classification algorithm, meta-heuristic isotonic separation with particle swarm optimization and convergence criterion (MeHeIS–CPSO), in which a particle swarm optimization-based meta-heuristic is embedded in the training phase to find a solution for LPP. The proposed framework formulates the LPP as a directed acyclic graph (DAG) and arranges decision variables using topological sort. It obtains a new threshold value from training set and sets up a convergence criterion using this threshold. It also deploys a new correlation coefficient-based supervised feature selection technique to select isotonic features and improves predictive accuracy of the classifier. Experiments are conducted on publicly available data sets and synthetic data set. Theoretical, empirical, and statistical analyses show that MeHeIS–CPSO is superior to its predecessors in terms of training time and predictive ability on large data sets. It also outperforms state-of-the-art machine learning and isotonic classification techniques in terms of predictive performance on small- and large-scale data sets.

RETRACTED ARTICLE: Tolerance rough set firefly-based quick reduct

In medical information system, there are a lot of features and the relationship among elements is solid. In this way, feature selection of medical datasets gets awesome worry as of late. In this article, tolerance rough set firefly-based quick reduct, is developed and connected to issue of differential finding of diseases. The hybrid intelligent framework intends to exploit the advantages of the fundamental models and, in the meantime, direct their restrictions. Feature selection is procedure for distinguishing ideal feature subset of the original features. A definitive point of feature selection is to build the precision, computational proficiency and adaptability of expectation strategy in machine learning, design acknowledgment and information mining applications. Along these lines, the learning framework gets a brief structure without lessening the prescient precision by utilizing just the chose remarkable features. In this research, a hybridization of two procedures, tolerance rough set and as of late created meta-heuristic enhancement calculation, the firefly algorithm is utilized to choose the conspicuous features of medicinal information to have the capacity to characterize and analyze real sicknesses. The exploratory results exhibited that the proficiency of the proposed system outflanks the current supervised feature selection techniques.

A novel breast tumor classification algorithm using neutrosophic score features

A lot of studies confirmed the seriousness of breast cancer as the most tumors lethal to women worldwide. Early detection and diagnosis of breast cancer are of great importance to increase treatment options and patients’ survival rate. Ultrasound is one of the most frequently used methods to detect and diagnosis breast tumor due to its harmlessness and inexpensiveness. However, problems were found in the tumor diagnosis and classification as benign and malign on ultrasound image for its vagueness, such as speckle noise and low contrast. In this paper, we propose a novel breast tumor classification algorithm that combines texture and morphologic features based on neutrosophic similarity score. Then, a supervised feature selection technique is employed to reduce feature space. Finally, a support vector machine (SVM) classifier is employed to prove the discrimination power of the proposed features set. The proposed system is validated by 112 cases (58 malign, 54 benign). The experimental results show that such features set is promising and 99.1% classification accuracy is achieved.

Self-adaptive differential evolution for feature selection in hyperspectral image data

Hyperspectral images are captured from hundreds of narrow and contiguous bands from the visible to infrared regions of electromagnetic spectrum. Each pixel of an image is represented by a vector where the components of the vector constitute the reflectance value of the surface for each of the bands. The length of the vector is equal to the number of bands. Due to the presence of large number of bands, classification of hyperspectral images becomes computation intensive. Moreover, higher correlation among neighboring bands increases the redundancy among them. As a result, feature selection becomes very essential for reducing the dimensionality. In the proposed work, an attempt has been made to develop a supervised feature selection technique guided by evolutionary algorithms. Self-adaptive differential evolution (SADE) is used for feature subset generation. Generated subsets are evaluated using a wrapper model where fuzzy k-nearest neighbor classifier is taken into consideration. Our proposed method also uses a feature ranking technique, ReliefF algorithm, for removing duplicate features. To demonstrate the effectiveness of the proposed method, investigation is carried out on three sets of data and the results are compared with four other evolutionary based state-of-the-art feature selection techniques. The proposed method shows promising results compared to others in terms of overall classification accuracy and Kappa coefficient.