Attribute Selection Method Research Articles

Predictive models assessment based on CRISP-DM methodology for students performance in Colombia - Saber 11 Test

The purpose of this paper is to evaluate several machine learning models under the CRISP-DM methodology in order to determine, through its metrics, the best model for predicting the performance of high school students in the Colombian Caribbean region in the Saber 11º test, while proposing a new methodology for evaluating the results of the test by regions in order to take into account the socioeconomic particularities of each one of them. The CRISP-DM methodology is taken as a basis due to its maturity, this methodology allows the extraction of business and data knowledge, offers a guide for data preparation, modeling and validation of the models; it is expected that the proposed methodology will be implemented by the Colombian Institute for the Promotion of Higher Education (ICFES), departmental education secretariats and educational institutions. A variety of techniques and tools were used to develop ETL processes to obtain a data set with the most relevant attributes, in order to evaluate four machine learning models developed with the J48 (C4.5), LMT, PART and Multilayer Perceptron algorithms; obtaining that the best data set and the best learning model is obtained using the InfoGain attribute selection method and the LMT decision tree algorithm, respectively. Therefore, this project will facilitate the actors of the National Education System to make decisions for the benefit of students and the quality of education in the country, especially in the Caribbean region.

Context‐awaretext classification system to improve the quality of text: A detailed investigation and techniques

SummaryText classification is one of the most important tasks to extract information from the Internet and identifying the best text representation settings. With the increase of data volume on the world wide web, the significance of text classification increases. This situation requires huge human efforts to understand and classify the digital data available on the Internet. Text classification is classifying the number of text files into different classes. The data or text available on the Internet is in an unstructured form which increases the difficulty to understand and classify it for useful purposes. This paper proposes a context‐aware text classification system to improve text quality. We use a content‐aware recommendation system to extract the data from well‐known news databases. Text preprocessing techniques like tokenization, stemming, and stop words removal are studied in detail. Furthermore, unigram, bigram, and trigram attributes are also being tested. Attribute selection methods are also examined and their impact on the text classification results. To carry out a detailed investigation, 11 versions are created of each dataset to save the time in experimentation process and applied the different preprocessing techniques to understand the impact of each technique on classification results. The proposed system is compared with the existing approach to check the accuracy where the proposed system achieved better performance.

Feature Selection Method Based on Grey Wolf Optimization and Simulated Annealing

Background: Feature selection sometimes also known as attribute subset selection is a process in which optimal subset of features are elected with respect to target data by reducing dimensionality and removing irrelevant features. There will be 2n possible solutions for a dataset having n number of features that is difficult to solve by conventional attribute selection method. In such cases metaheuristic-based methods generally outruns the conventional methods. Objective: The main aim of this paper is to enhance the classification accuracy and minimize the number of selected features and error rate. Methods: To achieve the objective, a binary metaheuristic feature selection method bGWOSA based on grey wolf optimization and simulated annealing has been introduced. The proposed feature selection method uses simulated annealing for equalizing the trade-off between exploration and exploitation. The performance of the proposed binary feature selection method has been examined on the ten feature selection benchmark datasets taken from UCI repository and compared with binary cuckoo search, binary particle swarm optimization, binary grey wolf optimization, binary bat algorithm and binary hybrid whale optimization method. Results: The proposed feature selection method achieves the highest accuracy for the most of datasets compared to state-of-the-art. Further, from the experimental and statistical results, efficacy of the proposed feature selection method has been validated. Conclusion: Classification accuracy can be enhanced by employing feature selection methods. Moreover, performance can also be enhanced by tuning the control parameters of metaheuristic methods.

A financial statement fraud model based on synthesized attribute selection and a dataset with missing values and imbalanced classes

Many financial fraud events have occurred over the past few decades. These events have led to massive losses for investors. Hence, government officials have begun to focus on the problem and have issued several decrees (acts) on financial fraud. Many scholars have explored the factors of financial fraud, and although their results performed well, most studies have not generated a set of useful rules to support auditors. Furthermore, data on financial statement fraud usually constitute an imbalanced class problem, and previous work minimally addresses this problem. Therefore, this study, based on the handling of missing values and imbalanced classes, builds a detecting model of financial statement fraud. First, it utilizes listwise and pairwise deletion to remove missing values. Second, it proposes three merged attribute selection methods and applies a nonlinear distance correlation to select important attributes. Third, it applies undersampling and oversampling to address the imbalanced classes. Finally, it uses rule-based classifiers to generate a set of useful rules. In practice, this study employs a list of fraudulent companies to collect data on financial statement fraud. We summarize the results as follows: (1) the pairwise deletion removes fewer records than does listwise removal in handling missing values; (2) the merged attribute selection (Com_I4) has the best performance on the four evaluation criteria; (3) the oversampling can enhance accuracy, and has the lowest type 1 and type 2 errors; (4) the random forest of Com_I4 can build the optimal model of financial statement fraud in the pairwise deletion and random oversampling; and (5) the results show that the ensemble learning (random forest) is a robust model in this study. Finally, these results in this study can be provided to practitioners, investors, and auditing personnel as references.

Sağlık Veri Setlerinde Öznitelik Seçiminin Sınıflandırma Performansına Etkisi

Nowadays, since data sets become very high-dimensional and specific with the data collected from different devices, attribute selection has an important pre-processing task in reducing data size in data mining. This study aims to improve classification performance by reducing the calculation time and cost by using attribute selection methods. Attribute selection methods are examined under three main headings; filter method, wrapper method, and embedded method. In the study, support vector machine, Naive Bayes and decision trees methods (J48) among the machine learning classification algorithms were used. Data sets were obtained from UCI and Kaggle databases. Accuracy values were calculated to compare the classification performances of the algorithms. WEKA version 3.8.3, R3.3.0 and Tableu programs were performed in all analyzes. After unnecessary features were extracted by using appropriate methods in the analysis; classification performances and run times of algorithms were calculated. Accuracy values increased to 87% for Colorectal Histology MNIST, 85% for Parkinson's disease, 97% for SCADI, 100% for HCC, and 78% for breast cancer after attribute selection. The algorithm with the highest performance was found as a wrapper method with decision trees (J48). While the fastest algorithm was filter method, the longest-running algorithm was the wrapper method. According to results, the performance improvement was higher in feature sets with a large number of attributes after selecting feature. As a result, low dimensional data sets may provide higher classification accuracy with lower calculation costs.

Computational intelligence techniques for automatic detection of Wi-Fi attacks in wireless IoT networks

These days, number of smart products based on Internet-of-Things (IoT) has been increased. These products are unified via various wireless technologies like, Bluetooth, Z-wave, Wi-Fi, Zigbee, etc. While the need on the wireless networks has improved, the assaults against them throughout the time have expanded on top. In order to identify these assaults, an intrusion detection system (IDS) with a prominent precision and low identification time is required. In this work, a machine learning (ML) based wireless intrusion detection system (WIDS) for wireless networks to effectively identify assaults against them has been proposed. A ML prototype has been implemented to categorize the wireless network records into ordinary or one of the particular assault categories. The operation of an IDS is extensively enhanced when the attributes are more discriminative and delegate. Different attribute selection methods have been investigated to identify the best set of attributes for the WIDS. The proposed model is evaluated on aegean wireless intrusion dataset using various parameters like attack detection rate, detection time, precision, F-measure, etc. The experimental evaluation is carried out in the tools like, Weka, Rstudio and Anaconda Navigator Python. Finally, the experimental result shows the best performing ML algorithm with best set of reduced attributes.

A novel permission-based Android malware detection system using feature selection based on linear regression

With the developments in mobile and wireless technology, mobile devices have become an important part of our lives. While Android is the leading operating system in market share, it is the platform most targeted by attackers. Although many solutions have been proposed in the literature for the detection of Android malware, there is still a need for attribute selection methods to be used in Android malware detection systems. In this study, a machine learning-based malware detection system is proposed to distinguish Android malware from benign applications. At the feature selection stage of the proposed malware detection system, it is aimed to remove unnecessary features by using a linear regression-based feature selection approach. In this way, the dimension of the feature vector is reduced, the training time is decreased, and the classification model can be used in real-time malware detection systems. When the results of the study are examined, the highest 0.961 is obtained according to the F-measure metric by using at least 27 features.

INTELLIGENT HIERARCHIZATION OF SERVICE ELEMENTS CONTRIBUTING TO OVERALL CUSTOMER SATISFACTION „CASE OF AIR ALGERIE CUSTOMERS”

The present work is part of relational marketing, in particular consumer behavior in the field of service. Relationship marketing is based mainly on three pillars: quality, satisfaction, and loyalty. The choice of our conceptual model is based on an in-depth literature of previous studies, in view of the multiplicity of research work carried out in this field and the diversity of points of view of the researchers. The fundamental research question is: How to increase the level of customer satisfaction to retain them? The objective of this article is both to determine the explanatory dimensions of consumer satisfaction, but also to analyze their influence on overall consumer satisfaction. For this purpose, we have opted for the “FILTER” type attribute selection methods based on the “WEKA” programming environment. To this end, an empirical study was carried out. We employed quantitative methodology and data were analyzed by Structural Equation Models. The sample consisted of 242 customers of the airline “AIR ALGERIE”. The results obtained reveal to us what is hidden behind the overall customer satisfaction of this company. They also certify that the dimensions in question contribute differently to this satisfaction. Such results would help leaders to provide a double effort in order to bring real and effective improvements, but also to give more care to the relevant variables selected by the different methods applied. However, the results also revealed the importance of the service elements which contribute to overall customer satisfaction

Machine Learning Application for Predicting Autistic Traits in Toddlers

Preliminary diagnosis of medical conditions, such as autism spectrum disorder (ASD), requires an understanding of the influential traits, for example autistic traits, during the screening process. Therefore, selecting the right attributes is a critical part of model construction in medical applications such as ASD screening, as this directly impacts the accuracy and efficiency of classification. This research investigates different methods for selecting attributes, including Chi-square (CHI), correlation feature set, information gain, Gini index and fast correlated-based filter to identify highly impactful autistic traits using over 1000 data observations of cases and controls related to toddlers. We seek to find the common autistic traits that influence the pre-diagnosis process for ASD obtained by these attribute selection methods from a real autism dataset related to toddlers and their impact on the performance of the screening process. To achieve the aim, an empirical methodology involving the use of three classification algorithms, AdaBoost, k-Nearest Neighbour (kNN) and ID3, has been used to derive models from the various different datasets chosen prior to training according to the considered attribute selection methods. These models are evaluated using evaluation metrics including specificity, sensitivity, accuracy and area under curve. Empirical results using the classification techniques for different attribute sets for the toddlers’ dataset show which influential autistic traits can be utilized by clinicians and diagnosticians to speed up the pre-diagnosis process for ASD and to enhance classification performance. More importantly, we show which attribute selection methods identify the relevant attributes that influence the preliminary process for diagnosis.

Individual Attribute Selection Using Information Gain Based Distance for Group Classification of Elderly People With Hypertension

Attribute selection is the process of selecting relevant attributes being used in model construction to enhance model accuracy. For general medical oriented classification applications, classical attribute selection methods principally select common attributes in the dataset for all individuals. The idea of using individual attributes is proposed in this study to represent the difference among individuals for self-diagnosis. Consequently, this study proposes a new attribute selection method, called information gain based distance (IGD), for individual attribute selection, which represents an individual's health condition differently and can be used for effective classification. The proposed method combines the concept of information gain and objective distance to select individual attributes affecting classification. The IGD method is expected to provide higher classification performance than classical attribute selection methods. To assess the performance of the IGD method, classification accuracy between data with classical attribute selections and with the IGD method is compared. The case study is conducted with 971 secondary data used for group classification of elderly people with hypertension. The classification result of different classifiers was compared, including K-nearest neighbors, neural network, and naive Bayes. The comparison revealed that the classification of data with the IGD attribute selection method provided an average classification accuracy of 98.73%. In comparison, those classifications of data with classical attribute selection methods provided 62.99%, 62.99%, 62.62%, and 62.85% for information gain, Gini index, chi-squared, and decision tree, respectively. The results showed that data classification with the IGD method provided higher performance than those with the classical attribute selection methods.

Ciphertext-Policy Attribute Based Encryption with Selectively-Hidden Access Policy

In conventional Ciphertext-Policy Attribute-Based Encryption (CP-ABE), the access policy appears in plaintext form that might reveal confidential user information and violate user privacy. CP-ABE with hidden access policies hides all attributes, but the computational burden increases due to the attribute hiding. In this paper, we present a Linear Secret Sharing Scheme (LSSS) access structure CP-ABE scheme that hides only sensitive attributes, rather than all attributes, in the access policy. We also provide an attribute selection method to choose these sensitive attributes and use an Attribute Bloom Filter (ABF) to hide them. Compared with the existing major CP-ABE schemes with hidden access policies, our proposed scheme is flexible in selecting attributes to hide. This scheme enhances the efficiency of policy hiding while still protecting policy privacy. Test results show that our approach is reasonable and feasible.

Novel attribute selection technique for an efficient intrusion detection system

An intrusion detection system (IDS) is one of the popular countermeasures against network attacks or anomalies in the network. IDS requires data dimension reduction methods such as data filtering, attribute selection, and data clustering to improve the system's performance. Identifying the most significant attributes will produce the computational time complexity drastically. This paper proposes novel attribute selection methods for data reduction. The NSL-KDD dataset is selected for experimental evaluation. The attribute value ratios (AVRs) is calculated using average values with numeric attributes and frequency with binary attributes to select attributes. The proposed attributes selection model is also compared with the traditional attribute section method that is correlation-based feature selection (CFS), information gain (IG) and gain ratio (GR). Further, the accuracy (ACC), detection rate (DR) and false alarm rate (FAR) are computed using the proposed attributes selection model with J48Consolidated classifier and achieve better results than other state-of-the-art methodologies.

Associating Measles Vaccine Uptake Classification and its Underlying Factors Using an Ensemble of Machine Learning Models

Measles is one of the significant public health issues responsible for the high mortality rate around the globe, especially for developing countries. Using nationally representative demographic and health survey data, measles vaccine utilization has been classified, and its underlying factors are identified through an ensemble Machine Learning (ML) approach. Firstly, missing values are imputed employing various approaches, and then several feature selection techniques have been applied to identify the crucial attributes for predicting measles vaccination. A grid search hyperparameter optimization technique has been applied for tuning the critical hyperparameters of different ML models, such as Naive Bayes, random forest, decision tree, XGboost, and lightgbm. The categorization performance of the individual optimized ML model as all as their ensembles have been reported utilizing our proposed BDHS dataset. Individually, the optimized lightgbm provides the highest precision and AUC of 79.90% and 77.80%, respectively. This result improved when the optimized lightgbm is ensembled with XGboost, providing the precision and AUC of 84.60% and 80.0%, respectively. Our result reveals that the statistical median imputation technique with the XGboost-based attribute selection method and the lightgbm classifier provides the best individual result. The performance has been improved when the proposed weighted ensemble of the XGboost and lightgbm approach has been adapted with the same preprocessing and recommended for measles vaccine utilization. The significance of our proposed approach is that it utilizes minimum attributes collected from the child and their family members and yielded 80.0%accuracy, making it easily explainable by caregivers and healthcare personnel. Finally, our predictive model provides an early detection procedure to help national policymakers enforce new policies with specific rules and regulations. The data and source codes that support the findings of this study are available at https://github.com/kamruleee51/measles_vaccine_uptake.

A Safe-Region Imputation Method for Handling Medical Data with Missing Values

Medical data usually have missing values; hence, imputation methods have become an important issue. In previous studies, many imputation methods based on variable data had a multivariate normal distribution, such as expectation-maximization and regression-based imputation. These assumptions may lead to deviations in the results, which sometimes create a bottleneck. In addition, directly deleting instances with missing values may have several problems, such as losing important data, producing invalid research samples, and leading to research deviations. Therefore, this study proposed a safe-region imputation method for handling medical data with missing values; we also built a medical prediction model and compared the removed missing values with imputation methods in terms of the generated rules, accuracy, and AUC. First, this study used the kNN imputation, multiple imputation, and the proposed imputation to impute the missing data and then applied four attribute selection methods to select the important attributes. Then, we used the decision tree (C4.5), random forest, REP tree, and LMT classifier to generate the rules, accuracy, and AUC for comparison. Because there were four datasets with imbalanced classes (asymmetric classes), the AUC was an important criterion. In the experiment, we collected four open medical datasets from UCI and one international stroke trial dataset. The results show that the proposed safe-region imputation is better than the listing imputation methods and after imputing offers better results than directly deleting instances with missing values in the number of rules, accuracy, and AUC. These results will provide a reference for medical stakeholders.

Real-time dynamic security analysis of power systems using strategic PMU measurements and decision tree classification

Fast and accurate online dynamic security analysis (DSA) is the key enabler for secure operation of modern power systems. Real-time assessment of the current power system operating state and increased awareness about plausible future insecurity can enable necessary operational and control measures to ensure secure operation. This paper proposes an ensemble decision tree (DT)-based online DSA method for large-scale interconnected power system networks using wide area measurement (WAMS) with phasor measurement units (PMU). A novel attribute selection method has been demonstrated for optimizing PMU installation at strategic buses in large-scale power networks. Multi-stage screening of the initial measurements has been done to minimize the data acquisition cost and computation overhead, which are the key challenges in real-time DSA. The ensemble DT classifier was trained offline using data from the operational model of the power system under different system loading and contingency conditions. The trained classifier provides online security assessment and classifies the power system’s current operating state as secure or insecure based on real-time measurements of the key attributes by selective PMUs. The proposed scheme was tested on IEEE 118-bus system, and the results demonstrate that it has the potential to be used as a reliable online DSA method.

Data mining for sex estimation based on cranial measurements

The aim of the present study is to develop effective and understandable classification models for sex estimation and to identify the most dimorphic linear measurements in adult crania by means of data mining techniques. Furthermore, machine learning models and models developed through logistic regression analysis are compared in terms of performance. Computed tomography scans of 393 adult individuals were used in the study. A landmark-based approach was applied to collect the metric data. The three-dimensional coordinates of 47 landmarks were acquired and used for calculation of linear measurements. Two datasets of cranial measurements were assembled, including 37standard measurements and 1081 interlandmark distances, respectively. Three data mining algorithms were applied: the rule induction algorithms JRIP and Ridor, and the decision tree algorithm J48. Two advanced attribute selection methods (Weka BestFirst and Weka GeneticSearch) were also used. The best accuracy result (91.9 %) was achieved by a set of rules learnt by the JRIP algorithm from the dataset constructed by application of the GeneticSearch selection algorithm to the dataset of standard cranial measurements. The set consisted of five rules including seven cranial measurements. Its accuracy was even better than the classification rates achieved by the logistic regression models. Concerning the second dataset of nonstandard measurements, the best accuracy (88.3 %) was obtained by using classification models learnt by two algorithms – JRIP with a dataset preprocessed by the BestFirst selection algorithm and Ridor with preprocessing by the GeneticSearch selection algorithm. Our experiments show that for the two datasets mentioned above the rule-based models contain smaller sets of rules with shorter lists of measurements and achieve better classification accuracy results in comparison with decision tree-based models.

Multi-models in predicting RNA solvent accessibility exhibit the contribution from none-sequential attributes and providing a globally stable modeling strategy

Recently, multiple researches of solvent accessibility on protein are well established. However, research on solvent accessibility of RNA faces a few challenges such as instability and diversity of RNA tertiary structure. Nowadays, no study has examined the predicting performance from different datasets built from multi-models and multi-attributes, but it is an important part of measuring the overall performance of modeling. Therefore, we performed a comprehensive comparison for predicting RNA solvent accessibility based on two datasets. 15,923 (12,229 ​+ ​3694) samples and 512(336 ​+ ​176) attributes were generated for attribute selection, finally 336 models were built for predicting. 12 modeling methods and 2 attribute selection methods were used for modeling and evaluating. This work provided a strategy for getting stable expectation when predicting RNA solvent accessibility. These results would be useful in further experimental or computational design for RNA solvent accessibility predicting and we hope this work could help the related researches which need a workflow for stable prediction.

Big data attribute selection method in distributed network fault diagnosis database

Big data attribute selection method in distributed network fault diagnosis database

Performance comparison of nonlinear and linear regression algorithms coupled with different attribute selection methods for quantitative structure - retention relationships modelling in micellar liquid chromatography

In micellar liquid chromatography (MLC), the addition of a surfactant to the mobile phase in excess is accompanied by an alteration of its solubilising capacity and a change in the stationary phase's properties. As an implication, the prediction of the analytes’ retention in MLC mode becomes a challenging task. Mixed Quantitative Structure – Retention Relationships (QSRR) modelling represents a powerful tool for estimating the analytes’ retention.This study compares 48 successfully developed mixed QSRR models with respect to their ability to predict retention of aripiprazole and its five impurities from molecular structures and factors that describe the Brij - acetonitrile system. The development of the models was based on an automatic combining of six attribute (feature) selection methods with eight predictive algorithms and the optimization of hyper-parameters. The feature selection methods included Principal Component Analysis (PCA), Non-negative Matrix Factorization (NMF), ReliefF, Multiple Linear Regression (MLR), Mutual Info and F-Regression. The series of investigated predictive algorithms comprised Linear Regressions (LR), Ridge Regression, Lasso Regression, Artificial Neural Networks (ANN), Support Vector Regression (SVR), Random Forest (RF), Gradient Boosted Trees (GBT) and K-Nearest neighbourhood (k-NN).A sufficient amount of data for building the model (78 cases in total) was provided by conducting 13 experiments for each of the 6 analytes and collecting the target responses afterwards. Different experimental settings were established by varying the values of the concentration of Brij L23, pH of the aqueous phase and acetonitrile content in the mobile phase according to the Box-Behnken design. In addition to the chromatographic parameters, the pool of independent variables was expanded by 27 molecular descriptors from all major groups (physicochemical, quantum chemical, topological and spatial structural descriptors). The best model was chosen by taking into consideration the Root Mean Square Error (RMSE) and cross-validation (CV) correlation coefficient (Q2) values.Interestingly, the comparative analysis indicated that a change in the set of input variables had a minor impact on the performance of the final models. On the other hand, different regression algorithms showed great diversity in the ability to learn patterns conserved in the data. In this regard, testing many regression algorithms is necessary in order to find the most suitable technique for model building. In the specific case, GBT-based models have demonstrated the best ability to predict the retention factor in the MLC mode. Steric factors and dipole-dipole interactions have proven to be relevant to the observed retention behaviour. This study, although being of a smaller scale, is a most promising starting point for comprehensive MLC retention prediction.

SysDroid: a dynamic ML-based android malware analyzer using system call traces

Android is a popular open-source operating system highly susceptible to malware attacks. Researchers have developed machine learning models, learned from attributes extracted using static/dynamic approaches to identify malicious applications. However, such models suffer from low detection accuracy, due to the presence of noisy attributes, extracted from conventional feature selection algorithms. Hence, in this paper, a new feature selection mechanism known as selection of relevant attributes for improving locally extracted features using classical feature selectors (SAILS), is proposed. SAILS, targets on discovering prominent system calls from applications, and is built on the top of conventional feature selection methods, such as mutual information, distinguishing feature selector and Galavotti–Sebastiani–Simi. These classical attribute selection methods are used as local feature selectors. Besides, a novel global feature selection method known as, weighted feature selection is proposed. Comprehensive analysis of the proposed feature selectors, is conducted with the traditional methods. SAILS results in improved values for evaluation metrics, compared to the conventional feature selection algorithms for distinct machine learning models, developed using Logistic Regression, CART, Random Forest, XGBoost and Deep Neural Networks. Our evaluations observe accuracies ranging between 95 and 99% for dropout rate and learning rate in the range 0.1–0.8 and 0.001–0.2, respectively. Finally, the security evaluation of malware classifiers on adversarial examples are thoroughly investigated. A decline in accuracy with adversarial examples is observed. Also, SAILS recall rate of classifier subjected to such examples estimate in the range of 24.79–92.2%. However, prior to the attack, the true positive rate obtained by the classifier is reported between 95.2 and 99.79%. The results suggest that the hackers can bypass detection, by discovering the classifier blind spots, on augmenting a small number of legitimate attributes.