Classification Task Of Data Mining Research Articles

Secure Two-Party Decision Tree Classification Based on Function Secret Sharing

Decision tree models are widely used for classification tasks in data mining. However, privacy becomes a significant concern when training data contain sensitive information from different parties. This paper proposes a novel framework for secure two-party decision tree classification that enables collaborative training and evaluation without leaking sensitive data. The critical techniques employed include homomorphic encryption, function secret sharing (FSS), and a custom secure comparison protocol. Homomorphic encryption allows computations on ciphertexts, enabling parties to evaluate an encrypted decision tree model jointly. FSS splits functions into secret shares to hide sensitive intermediate values. The comparison protocol leverages FSS to securely compare attribute values to node thresholds for tree traversal, reducing overhead through efficient cryptographic techniques. Our framework divides computation between two servers holding private data. A privacy-preserving protocol lets them jointly construct a decision tree classifier without revealing their respective inputs. The servers encrypt their data and exchange function secret shares to traverse the tree and obtain the classification result. Rigorous security proofs demonstrate that the protocol protects data confidentiality in a semihonest model. Experiments on benchmark datasets confirm that the approach achieves high accuracy with reasonable computation and communication costs. The techniques minimize accuracy loss and latency compared to prior protocols. Overall, the paper delivers an efficient, modular framework for practical two-party secure decision tree evaluation that advances the capability of privacy-preserving machine learning.

Research on a Decision Tree Classification Algorithm Based on Granular Matrices

The decision tree is one of the most important and representative classification algorithms in the field of machine learning, and it is an important technique for solving data mining classification tasks. In this paper, a decision tree classification algorithm based on granular matrices is proposed on the basis of granular computing theory. Firstly, the bit-multiplication and bit-sum operations of granular matrices are defined. The logical operations between granules are replaced by simple multiplication and addition operations, which reduces the operation time. Secondly, the similarity between granules is defined, the similarity metric matrix of the granular space is constructed, the classification actions are extracted from the similarity metric matrix, and the classification accuracy is defined by weighting the classification actions with the probability distribution of the granular space. Finally, the classification accuracy of the conditional attribute is used to select the splitting attributes of the decision tree as the nodes to form forks in the tree, and the similarity between granules is used to judge whether the data types in the sub-datasets are consistent to form the leaf nodes. The feasibility of the algorithm is demonstrated by means of case studies. The results of tests conducted on six UCI public datasets show that the algorithm has higher classification accuracy and better classification performance than the ID3 and C4.5.

An outliers detection and elimination framework in classification task of data mining

An outlier is a datum that is far from other data points in which it occurs. It can have a considerable impact on the output. Therefore, removing or resolving it before the analysis is essential to prevent skewing. Outliers in a survey sampling can have a significant outcome on statistical results. The goal of discovering outliers in data mining is to find a pattern in data that does not conform to expected behavior. In this paper, we have proposed a framework in which a popular statistical approach termed Inter-Quartile Range (IQR) is used to detect outliers in data and deal with them by Winsorizing method. A radial basis function network trained by teaching a learning-based optimization model is developed using the preprocessed dataset under this framework. A few standard University of California Irvine (UCI)datasets are employed to measure the framework’s effectiveness. The outcome of the experiments shows that our proposed framework can be a viable alternative tool for the classification task of data mining where prior outliers preprocessing is necessary.

Classification rule mining based on Pareto-based Multiobjective Optimization

This paper introduces a novel classification rule mining model based on Pareto-based Multiobjective Optimization called CRM-PM. The process of rule extraction is a challenging classification task in data mining since it has several constraints and conflicting objectives such as accuracy and comprehensibility. In this study, this task is accepted as a multi-objective optimization problem. Classification accuracy and misclassification ratio are assigned as evaluation criteria. The candidate solutions are generated in the direction of a proposed strategy to determine optimal ranges of the attributes that form the rules. The proposed approach is applied on eight benchmark datasets (Iris Plants, Wine Quality, Glass Identification, Stat log (Heart), Haberman’s Survival, E-coli, Wisconsin Breast Cancer, and Pima Indians Diabetes) included in the University of California at Irvine machine learning repository. Furthermore, CRM-PM is run in three different validation modes: cross-validation, training without test data, and training with random splitting. Regarding experimental results, it can be said that the presented method has a promising capability for classification, and it achieves comparative or superior results.

Improvement of insurance agents performance using data mining in OV agency

One problem that occurs in the insurance industry is the high turnover rate of insurance agents. It makes many companies must pay a large fee for the recruitment and training of new agents. OV Agency had high insurance agent turnover rate. This company wants to improve the agents performance to avoid the turnover. This research aims to identify prospective insurance agents who have better performance at recruitment process by using classification task of data mining. There are three classification methods were used . They were decision tree (J48), Neural Network (MLP) and Naive Bayes. Based on the result, MLP has the best average accuracy value 84.2% and ROC area value 0.87 (good classification). MLP was chosen to predict the 25 insurance agent candidates and 3 agents were selected because they entered class A (better performance) in all performance criteria. The prediction results show that three agents are men, aged 25-32 years old, divorced, senior high school education and master degree, work status in other companies, and have work experience of 0-3 years. The OV Agency can consider these result in new agent recruitment process.

Annealing strategy for an enhance rule pruning technique in ACO-Based rule classification

<span>Ant colony optimization (ACO) was successfully applied to data mining classification task through ant-mining algorithms. Exploration and exploitation are search strategies that guide the learning process of a classification model and generate a list of rules. Exploitation refers to the process of intensifying the search for neighbors in good regions, </span><span>whereas exploration aims towards new promising regions during a search process. </span><span>The existing balance between exploration and exploitation in the rule construction procedure is limited to the roulette wheel selection mechanism, which complicates rule generation. Thus, low-coverage complex rules with irrelevant terms will be generated. This work proposes an enhancement rule pruning procedure for the ACO algorithm that can be used in rule-based classification. This procedure, called the annealing strategy, is an improvement of ant-mining algorithms in the rule construction procedure. Presented as a pre-pruning technique, the annealing strategy deals first with irrelevant terms before creating a complete rule through an annealing schedule. The proposed improvement was tested through benchmarking experiments, and results were compared with those of four of the most related ant-mining algorithms, namely, Ant-Miner, CAnt-Miner, TACO-Miner, and Ant-Miner with hybrid pruner. </span><span>Results display that our proposed technique achieves better performance in terms of classification accuracy, model size, and </span><span>computational time. </span><span>The proposed annealing schedule can be used in other ACO variants for different applications to improve classification accuracy.</span>

Filter Variable Selection Algorithm Using Risk Ratios for Dimensionality Reduction of Healthcare Data for Classification

This research developed and tested a filter algorithm that serves to reduce the feature space in healthcare datasets. The algorithm binarizes the dataset, and then separately evaluates the risk ratio of each predictor with the response, and outputs ratios that represent the association between a predictor and the class attribute. The value of the association translates to the importance rank of the corresponding predictor in determining the outcome. Using Random Forest and Logistic regression classification, the performance of the developed algorithm was compared against the regsubsets and varImp functions, which are unsupervised methods of variable selection. Equally, the proposed algorithm was compared with the supervised Fisher score and Pearson’s correlation feature selection methods. Different datasets were used for the experiment, and, in the majority of the cases, the predictors selected by the new algorithm outperformed those selected by the existing algorithms. The proposed filter algorithm is therefore a reliable alternative for variable ranking in data mining classification tasks with a dichotomous response.

A data mining application in credit scoring processes of small and medium enterprises commercial corporate customers

The constant need to assess loans makes risk evaluation a very important problem for the banking sector. A crucial function of the banks is to fund households and companies from various industries in the economy. Risk is taken by the banks as soon as a loan is given to an entity. Currently, there are sector‐and‐experience based methods of analysis employed by the banks to estimate the risks to be taken. For the credit process, there exist a large number of studies in the literature on scoring individual clients but there are very few studies on scoring small and medium enterprises (SME) commercial corporate customers. In this study, we propose an objective risk measurement method for the lending process of SME commercial corporate customers and performed classification task of data mining by collecting current customer data on credit evaluation process of a bank. For this purpose, we first create a risk measure by looking into the risks identified for existing customers by the analysts of a bank. These scores are used as target variable in the classification process. Then, we extract rules for estimating these scores using Weka software. We used six different algorithms, and compared results in terms of test accuracy, the number of rules, recall, precision and Kappa statistic. We obtained high accuracy rates on real life data by our approach. As a result, we showed that an objective evaluation strategy is possible to use in the lending process for SME commercial corporate customers in the banking system using data mining.This article is categorized under: Application Areas > Business and Industry Technologies > Classification Application Areas > Industry Specific Applications

Integrating fitness predator optimizer with multi-objective PSO for dynamic partitional clustering

Clustering is an unsupervised classification task in data mining, which partitions data set into certain number of clusters. Though many metaheuristics have been suggested to optimize clustering results, most of them are marred by the three key issues. First, clustering performance is devised on a single validity measure; hence, it produces a single best solution biased toward the criterion; it is unacceptable as clustering has multiple conflicting criteria. Second, they find difficulty in avoiding local optima owing to lack of balancing in exploration and exploitation in the search space. Third, they require number of clusters a priori which is difficult to decide in the real-world problems. To deal with the first issue, we follow Pareto-based approach to obtain diverse trade-off solutions by optimizing conceptually contradicting validity measures sum of squared error and connectedness. The well-known swarm intelligence-based metaheuristic particle swarm optimization (PSO) has a severe drawback that it converges prematurely owing to single directional information sharing mechanism among particles in the swarm. Therefore, for the second issue, we introduce fitness predator optimizer (FPO) to enhance diversity in the PSO in multi-objective optimization scenario; it is named as FPO-MOPSO. To address the third issue, we use dynamic clustering method to decide an optimum number of clusters (k) in the range of 2 to \(\sqrt{n}\), where n is number of objects in the data set. We use eight real-world and two artificial data sets for comparison of the FPO-MOPSO results with three well-known competitive methods MOPSO, MABC, and NSGA-II using four quality measures convergence, diversity, coverage, and ONVG. The obtained results show superiority of the FPO-MOPSO over its competitors. Two external validity indices of clustering results F-measure and Rand index also establish the above finding. As the decision maker picks only a single solution from the set of trade-off solutions, we employ an additional measure silhouette index to select a final single solution from the archive and compare it with known number of clusters in the data sets.

Intelligent Hypothermia Care System using Ant Colony Optimization for Rules Prediction

Intelligent Hypothermia Care System (IHCS) is an intelligence system uses set of methodologies, algorithms, architectures, and processes to determine where patients in a postoperative recovery area must be sent. Hypothermia is a significant concern after surgery. This paper utilizes the classification task in data mining to propose an intelligent technique to predict where to send a patient after surgery: intensive care unit, general floor or home. To achieve this goal, this paper evaluates the performance of decision tree algorithm, exemplifying the deterministic approach, against the AntMiner algorithm, exemplifying the heuristic approach, to choose the best approach in detecting the patient’s status. Results show the outperformance of the heuristic approach. The implication of this proposal will be twofold: in hypothermia treatment and in the application of ant colony optimization.

A Multilayer Perceptron Artificial Neural Networks Based a Preprocessing and Hybrid Optimization Task for Data Mining and Classification

Artificial neural networks (ANNs) optimization represent an attractive area that attract many researchers in different disciplines, this in the aim to improve the performance of this model. In literature, there is no fix theory that illustrates how to construct this non linear model. Thus, all proposed construction was based on empirical illustration. Multilayer perceptron (MLP) is one of the most used models in ANNs area. It was described as a good non linear approximator with a power ability to lean well non linear system, and most of research was limited to a 3 layers MLP, by describing that 3 layers are sufficient to have good approximation. In this context we are interested to this model construction for solving supervised classification tasks in data mining. This construction requires a preprocessing phase that seems to scribe be important for the final performance. This paper present a process of MLP construction based on two phases: a preparation phase and an optimization phase. The first one describes a process of data cleaning, discretization, normalization, expansion, reduction and features selection. The second phase aims to optimize the set of weights based on some combination of hybrid algorithms such back-propagation algorithm, a local search and different evolution. An empirical illustration will be done to in order to validate the proposed model. At the end, a comparison with others known classifiers will be done to justify the validity of the proposed model.

A hybrid feature selection method based on instance learning and cooperative subset search

The problem of selecting the most useful features from thousands of candidates in a low sample size data set arises in many areas of modern sciences. Feature subset selection is a key problem in such data mining classification tasks. In practice, it is very common to use filter methods. However, they ignore the correlations between genes which are prevalent in gene expression data. On the other hand, standard wrapper algorithms cannot be applied because of their complexity. Additionally, existing methods are not specially conceived to handle the small sample size of the data which is one of the main causes of feature selection instability. In order to deal with these issues, we propose a new hybrid, filter wrapper, approach based on instance learning. Its main challenge is that it converts the problem of the small sample size to a tool that allows choosing only a few subsets of features in a filter step. A cooperative subset search, CSS, is then proposed with a classifier algorithm to represent an evaluation system of wrappers. Our method is experimentally tested and compared with state-of-the-art algorithms based on several high-dimensional low sample size cancer datasets. Results show that our proposed approach outperforms other methods in terms of accuracy and stability of the selected subset.

A Global-best Harmony Search based Gradient Descent Learning FLANN (GbHS-GDL-FLANN) for data classification

While dealing with real world data for classification using ANNs, it is often difficult to determine the optimal ANN classification model with fast convergence. Also, it is laborious to adjust the set of weights of ANNs by using appropriate learning algorithm to obtain better classification accuracy. In this paper, a variant of Harmony Search (HS), called Global-best Harmony Search along with Gradient Descent Learning is used with Functional Link Artificial Neural Network (FLANN) for classification task in data mining. The Global-best Harmony Search (GbHS) uses the concepts of Particle Swarm Optimization from Swarm Intelligence to improve the qualities of harmonies. The problem solving strategies of Global-best Harmony Search along with searching capabilities of Gradient Descent Search are used to obtain optimal set of weight for FLANN. The proposed method (GbHS-GDL-FLANN) is implemented in MATLAB and compared with other alternatives (FLANN, GA based FLANN, PSO based FLANN, HS based FLANN, Improved HS based FLANN, Self Adaptive HS based FLANN, MLP, SVM and FSN). The GbHS-GDL-FLANN is tested on benchmark datasets from UCI Machine Learning repository by using 5-fold cross validation technique. The proposed method is analyzed under null-hypothesis by using Friedman Test, Holm and Hochberg Procedure and Post-Hoc ANOVA Statistical Analysis (Tukey Test & Dunnett Test) for statistical analysis and validity of results. Simulation results reveal that the performance of the proposed GbHS-GDL-FLANN is better and statistically significant from other alternatives.

Predicting the Pro-Longevity or Anti-Longevity Effect of Model Organism Genes with New Hierarchical Feature Selection Methods.

Ageing is a highly complex biological process that is still poorly understood. With the growing amount of ageing-related data available on the web, in particular concerning the genetics of ageing, it is timely to apply data mining methods to that data, in order to try to discover novel patterns that may assist ageing research. In this work, we introduce new hierarchical feature selection methods for the classification task of data mining and apply them to ageing-related data from four model organisms: Caenorhabditis elegans (worm), Saccharomyces cerevisiae (yeast), Drosophila melanogaster (fly), and Mus musculus (mouse). The main novel aspect of the proposed feature selection methods is that they exploit hierarchical relationships in the set of features (Gene Ontology terms) in order to improve the predictive accuracy of the Naïve Bayes and 1-Nearest Neighbour (1-NN) classifiers, which are used to classify model organisms' genes into pro-longevity or anti-longevity genes. The results show that our hierarchical feature selection methods, when used together with Naïve Bayes and 1-NN classifiers, obtain higher predictive accuracy than the standard (without feature selection) Naïve Bayes and 1-NN classifiers, respectively. We also discuss the biological relevance of a number of Gene Ontology terms very frequently selected by our algorithms in our datasets.

The Application of the Ant Colony Decision Rule Algorithm on Distributed Data Mining

ABSTRACT Based on the deep research of Ant Colony Algorithm, an Ant Colony decision rule Algorithm is proposed for rules mining based on the thought of ant colony and rule mining of decision tree. Then the algorithm is compared with C4.5 and applied on the rules mining and the results are showed by simulation. INTRODUCTION In general, the goal of data mining is to extract knowledge from data. Data mining is an inter-disciplinary field, whose core is at the intersection of machine learning, statistics and databases (Quinlan, 1986). There are several data mining tasks, including classification, regression, clustering, dependence modeling, etc. (Quinlan, 1993). Each of these tasks can be regarded as a kind of problem to be solved by a data mining algorithm. Therefore, the first step in designing a data mining algorithm is to define which task the algorithm will address. In recent, there are many mining tools, such as neutral network, gene algorithm, decision trees, rule referring, to predict the future development[1], and they are able to help people to make good decisions. But there are some shortcomings in these methods, such as incomprehensive results, over-fit rules and difficulties in being applied on distributed simulation. In this paper we propose an Ant Colony Optimization (ACO) algorithm (Dorigo, Maniezzo & Colorni, 1996; Stutzle & Hoos, 1997; and de A. Silla & Ramalho, 2001) for the classification task of data mining. In this task the goal is to assign each case (object, record, or instance) to one class, out of a set of predefined classes, based on the values of some attributes (called predictor attributes) for the case. In the context of the classification task of data mining, discovered knowledge is often expressed in the form of IF-THEN rules, as follows: IF THEN . The rule antecedent (IF part) contains a set of conditions, usually connected by a logical conjunction operator (AND). In this paper we will refer to each rule condition as a term, so that the rule antecedent is a logical conjunction of terms in the form: IF term1 AND term2 AND ... Each term is a triple, such as . The rule consequent (THEN part) specifies the class predicted for cases whose predictor attributes satisfy all the terms specified in the rule antecedent. From a data mining viewpoint, this kind of knowledge representation has the advantage of being intuitively comprehensible for the user, as long as the number of discovered rules and the number of terms in rule antecedents are not large. In this paper, the ACDR is applied for resolving the distributed database mining. The simulation results show that the ACDR is an available and correct algorithm for distributed mining. THE ANT COLONY OPTIMIZATION ALGORITHM The ant colony optimization technique has emerged recently as a novel meta-heuristic belongs to the class of problem-solving strategies derived from natural (other categories include neural networks, simulated annealing, and evolutionary algorithms). The ant system optimization algorithm is basically a multi-agent system where low level interactions between single agents (i.e., artificial ants) result in a complex behavior of the whole ant colony. Ant system optimization algorithms have been inspired by colonies of real ants, which deposit a chemical substance (called pheromone) on the ground. It was found that the medium used to communicate information among individuals regarding paths, and used to decide where to go, consists of pheromone trails. A moving ant lays some pheromone (in varying quantities) on the ground, thus making the path by a trail of this substance. While an isolated ant moves essentially at random, an ant encountering a previously laid trail can detect it and decide with high probability to follow it, thus reinforcing the trail with its own pheromone. …

Streamed Sampling on Dynamic data as Support for Classification Model

Data mining process on dynamically changing data have several problems, such as unknown data size and changing of class distribution . Random sampling method commonly applied for extracting general synopsis from very large database. In this research, Vitter’s reservoir algorithm is used to retrieve k records of data from the database and put into the sample. Sample is used as input for classification task in data mining. Sample type is backing sample and it saved as table contains value of id, priority and timestamp. Priority indicates the probability of how long data retained in the sample. Kullback-Leibler divergence applied to measure the similarity between database and sample distribution. Result of this research is showed that continuously taken samples randomly is possible when transaction occurs. Kullback-Leibler divergence with interval from 0 to 0.0001, is a very good measure to maintain similar class distribution between database and sample. Sample results are always up to date on new transactions with similar class distribution. Classifier built from balance class distribution showed to have better performance than from imbalance one.

A Parallel Data Mining Method Based on Complex Network

This paper addresses the well-known classification task of data mining, where the objective is to predict the class which an example belongs to. a complex network is a networkwith non-trivial topological features—features that do not occur in simple networks such as lattices or random graphs but often occur in real graphs. We present results evaluating the performance of the hybrid method based on complex network.

A GRASP algorithm for fast hybrid (filter-wrapper) feature subset selection in high-dimensional datasets

Feature subset selection is a key problem in the data-mining classification task that helps to obtain more compact and understandable models without degrading (or even improving) their performance. In this work we focus on FSS in high-dimensional datasets, that is, with a very large number of predictive attributes. In this case, standard sophisticated wrapper algorithms cannot be applied because of their complexity, and computationally lighter filter-wrapper algorithms have recently been proposed. In this work we propose a stochastic algorithm based on the GRASP meta-heuristic, with the main goal of speeding up the feature subset selection process, basically by reducing the number of wrapper evaluations to carry out. GRASP is a multi-start constructive method which constructs a solution in its first stage, and then runs an improving stage over that solution. Several instances of the proposed GRASP method are experimentally tested and compared with state-of-the-art algorithms over 12 high-dimensional datasets. The statistical analysis of the results shows that our proposal is comparable in accuracy and cardinality of the selected subset to previous algorithms, but requires significantly fewer evaluations.

A granular agent evolutionary algorithm for classification

By inspiration of the granular evolutionary algorithm, a Granular Agent Evolutionary Classification (GAEC) algorithm for the classification task in data mining is proposed. The method uses the granular agent to denote the set of some examples that have similar attributions and the knowledge base guides the evolution of granular agent. Also some granular evolutionary operators are designed for classification problem. Assimilation operator, exchange operator, and differentiation operator reflect the competitive, cooperative and self-learning ability of agent, respectively. Finally, some classification rules are extracted from granular agents by some strategy to forecast the sort of new data. Empirical study contains UCI data sets, KDDCUP99 data sets and remote image recognition. The test results show that the algorithm has a good classification prediction, and only need a small price for the training time. In most UCI data sets, the performance of GAEC is better than G-NET, OCEC and C4.5, which have good performance. At the same time, some Gaussian White Noise attributes are added to these UCI data sets and the results show GACE has some anti-noise abilities. To test the scalability of GAEC, two functions along two dimensions, the number of training examples and the number of attributes are used. Also, GAEC are applied to some real world fields, intrusion detection system and remote sensing image recognition. The experiments for KDDCUP99 verify GAEC has capability to deal with massive data in real world and good predicting capability for unknown type data. At last, the accuracy rate of GAEC is also good for the remote sensing image recognition.

COMPATIBILITY AS A HEURISTIC FOR CONSTRUCTION OF RULES BY ARTIFICIAL ANTS

Ant Miner algorithms are used for the classification task of data mining. This paper proposes a new version of Ant Miner algorithm called CAnt Miner that considers compatibility between the attributes, when constructing a rule. In addition to a new heuristic function based on compatibility among attributes, we also introduce a strategy to dynamically stop adding terms in the rule antecedent part instead of using a user specified threshold that specifies when to stop adding terms. We compare the performance of proposed approach with other Ant Miner versions and C4.5 in nine public domain data set. The experiment results show that the proposed approach achieves higher accuracy rate then other approaches.