Data Mining Classification Problems Research Articles

Hybrid Ant Colony Optimization and Genetic Algorithm for Rule Induction

In this study, a hybrid rule-based classifier namely, ant colony optimization/genetic algorithm ACO/GA is introduced to improve the classification accuracy of Ant-Miner classifier by using GA. The Ant-Miner classifier is efficient, useful and commonly used for solving rule-based classification problems in data mining. Ant-Miner, which is an ACO variant, suffers from local optimization problem which affects its performance. In our proposed hybrid ACO/GA algorithm, the ACO is responsible for generating classification rules and the GA improves the classification rules iteratively using the principles of multi-neighborhood structure (i.e., mutation and crossover) procedures to overcome the local optima problem. The performance of the proposed classifier was tested against other existing hybrid ant-mining classification algorithms namely, ACO/SA and ACO/PSO2 using classification accuracy, the number of discovered rules and model complexity. For the experiment, the 10-fold cross-validation procedure was used on 12 benchmark datasets from the University California Irwine machine learning repository. Experimental results show that the proposed hybridization was able to produce impressive results in all evaluation criteria.

Data mining and image analysis using genetic programming

Genetic programming (GP) is an artificial intelligence technique that benefits from evolutionary computations allowing computers to solve problems automatically. In this paper, we present an optimized genetic-programming-based classifier that directly solves the multi-class classification problems in data mining and image analysis. A new fitness function is proposed for multiclass classification and brain tumor detection, which is validated by 10-fold cross validation. Instead of defining static thresholds as boundaries to differentiate between multiple labels, our work presents a method of classification where a GP system learns the relationships among experiential data and models them mathematically during the evolutionary process. We propose an optimized GP classifier based on a combination of pruning subtrees and a new fitness function. An orthogonal least squares algorithm is also applied in the training phase to create a robust GP classifier. Our approach has been assessed on six multiclass datasets and on a magnetic resonance imaging (MRI) brain image for tumor detection. The results of data classification for Iris, Wine, Glass, Pima, BUPA Liver and Balance Scale datasets are compared with existing algorithms. The high accuracy of brain tumor classification provided by our GP classifier confirms the strong ability of the developed technique for complicated classification problems. We compared our approach in terms of speed with previous GP algorithms as well. The analyzed results illustrate that the developed classifier produces a productive and rapid method for classification tasks that outperforms the previous methods for more challenging multiclass classification problems.

A novel switching function approach for data mining classification problems

Rule induction (RI) is one of the known classification approaches in data mining. RI extracts hidden patterns from instances in terms of rules. This paper proposes a logic-based rule induction (LBRI) classifier based on a switching function approach. LBRI generates binary rules by using a novel minimization function, which depends on simple and powerful bitwise operations. Initially, LBRI generates instance codes by encoding the dataset with standard binary code and then generates prime cubes (PC) for all classes from the instance codes by the proposed reduced offset method. Finally, LBRI selects the most effective PC of the current classes and adds them into the binary rule set that belongs to the current class. Each binary rule represents an If–Then rule for the rule induction classifiers. The proposed LBRI classifier is based on basic logic functions. It is a simple and effective method, and it can be used by intelligent systems to solve real-life classification/prediction problems in areas such as health care, online/financial banking, image/voice recognition, and bioinformatics. The performance of the proposed algorithm is compared to six rule induction algorithms; decision table, Ripper, C4.5, REPTree, OneR, and ICRM by using nineteen different datasets. The experimental results show that the proposed algorithm yields better classification accuracy than the other rule induction algorithms on ten out of nineteen datasets.

Exploiting Data Mining for Fast Inter Prediction Mode Decision in HEVC

HEVC has made significant progress in coding efficiency. Compared with the previous generation H.264/AVC video coding standard, HEVC can save 50% of code rate under the same visual quality. However, this drop in bit rate is accompanied by an increase in computational complexity, which poses a great challenge to the application and development of HEVC. After analyzing the great correlation between the partitioning mode of coding unit in the encoder and several parameters, the partitioning problem of the CU is modeled as a classification problem in data mining, and it is decided to use a decision tree to solve this classification problem. We implement the decision tree on the official software HM16.2 and test the algorithm on the test set. Experiments show that fast algorithm based on decision tree can improve the encoding speed effectively with little influence on the encoding performance.

The application of remote sensing image sea ice monitoring method in Bohai Bay based on C4.5 decision tree algorithm

In The Paper, the remote sensing monitoring of sea ice problem was turned into a classification problem in data mining. Based on the statistic of the related band data of HJ1B remote sensing images, the main bands of HJ1B images related with the reflectance of seawater and sea ice were found. On the basis, the decision tree rules for sea ice monitoring were constructed by the related bands found above, and then the rules were applied to Liaodong Bay area seriously covered by sea ice for sea ice monitoring. The result proved that the method is effective.

Solving nonlinear classification problems with black hole optimisation and higher order Jordan Pi-sigma neural network: a novel approach

Nature inspired optimisation algorithms have been on a leading focus of all researchers since last few decades. Successful applications of such algorithms for solving diversified problem domains make them more popular. This presented research is about to implement a novel nature inspired algorithm with higher order neural network for solving nonlinear classification problems of data mining. A recently developed nature inspired algorithm, black hole optimisation has been used to train the weights of a higher order Jordan Pi-sigma neural network and to optimise the performance of the network. The proposed black hole algorithm-based higher order neural network has been developed for solving the classification problems in data mining. The result of the proposed method is compared with some other benchmark meta-heuristic techniques like PSO, GA and found that the proposed method is superior to other approaches.

Extraction methods for uncertain inference rules by ant colony optimization

Abstract In recent years, the research on data mining methods has received increasing attention. In this paper, we design an uncertain system with the extracted uncertain inference rules to solve the classification problems in data mining. And then, two extraction methods integrated with ant colony optimization are proposed for the generation of the uncertain inference rules. Finally, two applications are given to verify the effectiveness and superiority of the proposed methods.

PENINGKATAN OPTIMASI SENTIMEN DALAM PELAKSANAAN PROSES PEMILIHAN PRESIDEN BERDASARKAN OPINI PUBLIK DENGAN MENGGUNAKAN ALGORITMA NAÏVE BAYES DAN PARICLE SWARM OPTIMIZATION

- The development of increasingly advanced IT in the process of presidential elections. When the Presidential election of 2014 yesterday has a lot of people use the phrase does not educate inappropriate to be delivered among the public. Pros and cons indeed occur among people are so warm that they pour on the internet. This happens because when getting warm diperbincangan 2014 presidential election yesterday happened pengkubu-kubuan two candidates. Society can not adjust the development of IT process well. Naive Bayes is widely used for classification problems in data mining and machine learning for its simplicity and accuracy of classification impressive. Naive Bayes classifier has been shown to be very effective to solve the problem of large scale for text categorization with high accuracy. In addition to having many capabilities mentioned above, however this method has a drawback in the assumptions that are difficult to fulfill, namely the independence of the feature. Particle Swarm Optimization (PSO) is an evolutionary computation technique which is able to produce globally optimal solution in the search space through the interaction of individuals in a swarm of particles. PSO is widely used to solve optimization problems as well as the feature selection. Accuracy is generated on Naive Bayes algorithm amounted to 63.85% and AUC by 0523, while Naive Bayes and Particle Swarm Optimmization with an accuracy of 71.15% and the AUC of 0.600. It can be concluded that the application of optimization can improve the accuracy of 63.85% to 71.15%. Naive Bayes Model and Particle Swarm Optimization can provide solutions to the problems of classification review of public opinion news of the election in order to more accurately and optimally. Keywords:Public Opinion, Classification, Naive Bayes, Particle Swarm Optimization, Text Mining.

Intrusion Detection and Classification Using Ant Colony Optimization Algorithm

Studies of ant colonies have contributed in abundance to the set of intelligent algorithms. The modeling of pheromone depositing by ants in their search for the shortest paths to food sources resulted in the development of shortest path optimization algorithms. Ant colony optimization (ACO) algorithms have been successfully applied to combinatorial optimization tasks especially to data mining classification problem. Internet and local networks have become everywhere. So organizations are increasingly employing various systems that monitor IT security breaches because intrusion events are growing day by day. Ant-based algorithms or ant colony optimization (ACO) algorithms can be applied to the data mining field to extract rule-based classifiers and have been applied successfully to combinatorial optimization problems. More recently, researches applied ACO to data mining classification problems, where they introduced a classification algorithm called Ant-Miner algorithm. The Ant-Miner algorithm is based on the behavior of ants in searching of food. The aim of this paper is to use an Ant Colony-based classification system (Ant_Miner algorithm) to extract a set of rules for detection and classification, and it obtained a hopeful classification accuracy.

Hybrid decision tree and naïve Bayes classifiers for multi-class classification tasks

In this paper, we introduce two independent hybrid mining algorithms to improve the classification accuracy rates of decision tree (DT) and naïve Bayes (NB) classifiers for the classification of multi-class problems. Both DT and NB classifiers are useful, efficient and commonly used for solving classification problems in data mining. Since the presence of noisy contradictory instances in the training set may cause the generated decision tree suffers from overfitting and its accuracy may decrease, in our first proposed hybrid DT algorithm, we employ a naïve Bayes (NB) classifier to remove the noisy troublesome instances from the training set before the DT induction. Moreover, it is extremely computationally expensive for a NB classifier to compute class conditional independence for a dataset with high dimensional attributes. Thus, in the second proposed hybrid NB classifier, we employ a DT induction to select a comparatively more important subset of attributes for the production of naïve assumption of class conditional independence. We tested the performances of the two proposed hybrid algorithms against those of the existing DT and NB classifiers respectively using the classification accuracy, precision, sensitivity-specificity analysis, and 10-fold cross validation on 10 real benchmark datasets from UCI (University of California, Irvine) machine learning repository. The experimental results indicate that the proposed methods have produced impressive results in the classification of real life challenging multi-class problems. They are also able to automatically extract the most valuable training datasets and identify the most effective attributes for the description of instances from noisy complex training databases with large dimensions of attributes.

Using Non-Additive Measure for Optimization-Based Nonlinear Classification

Over the past few decades, numerous optimization-based methods have been proposed for solving the classification problem in data mining. Classic optimization-based methods do not consider attribute interactions toward classification. Thus, a novel learning machine is needed to provide a better understanding on the nature of classification when the interaction among contributions from various attributes cannot be ignored. The interactions can be described by a non-additive measure while the Choquet integral can serve as the mathematical tool to aggregate the values of attributes and the corresponding values of a non-additive measure. As a main part of this research, a new nonlinear classification method with non-additive measures is proposed. Experimental results show that applying non-additive measures on the classic optimization-based models improves the classification robustness and accuracy compared with some popular classification methods. In addition, motivated by well-known Support Vector Machine approach, we transform the primal optimization-based nonlinear classification model with the signed non-additive measure into its dual form by applying Lagrangian optimization theory and Wolfes dual programming theory. As a result, 2n – 1 parameters of the signed non-additive measure can now be approximated with m (number of records) Lagrangian multipliers by applying necessary conditions of the primal classification problem to be optimal. This method of parameter approximation is a breakthrough for solving a non-additive measure practically when there are relatively small number of training cases available (mn－1). Furthermore, the kernel-based learning method engages the nonlinear classifiers to achieve better classification accuracy. The research produces practically deliverable nonlinear models with the non-additive measure for classification problem in data mining when interactions among attributes are considered.

Credit Assessment of Bank Customers by a Fuzzy Expert System Based on Rules Extracted from Association Rules

Credit assessment is a very typical classification problem in Data Mining. A type of classification technique that has attracted an increasing number of attempts in recent years is finding classification rules based on association rule mining techniques. This paper aims to contribute to this kind of research by classifying the bank's customers via association rules with the use of the APRIORI algorithm and CRISP-DM methodology and considering the Experts' opinions to filter the obtained rules and define the Membership functions for the considered criteria, finally a Fuzzy Expert System is designed based on the selected rules from association rules to specify the Credit Degree of banks' customers. The presented steps have been studied in an Iranian Bank as empirical study.

Mining classification rules for gene expression data using enhanced quickreduct fuzzy-rough feature selection and ant colony optimisation

In this paper, the mining classification rules for gene expression data of yeast S.Cerevisiae is carried out using enhanced quickreduct fuzzy-rough feature selection (EQRFR-FS) with ant colony optimisation (ACO). Rough sets theory deals with uncertainty and vagueness of an information system in data mining. The ACO algorithms have been applied to combinatorial optimisation problems and data mining classification problems. The analyses are carried out in two phases. In the first phase, the features are extracted from the gene expression data and then feature selection is carried out from the extracted features using EQRFR-FS algorithm. In the second phase, from reduct set the classification rules mining are carried out with ant-miner algorithm. The dataset is obtained from the expression profiles maintained by Brown’s group at Stanford University.

A New Data Mining Scheme Using Artificial Neural Networks

Classification is one of the data mining problems receiving enormous attention in the database community. Although artificial neural networks (ANNs) have been successfully applied in a wide range of machine learning applications, they are however often regarded as black boxes, i.e., their predictions cannot be explained. To enhance the explanation of ANNs, a novel algorithm to extract symbolic rules from ANNs has been proposed in this paper. ANN methods have not been effectively utilized for data mining tasks because how the classifications were made is not explicitly stated as symbolic rules that are suitable for verification or interpretation by human experts. With the proposed approach, concise symbolic rules with high accuracy, that are easily explainable, can be extracted from the trained ANNs. Extracted rules are comparable with other methods in terms of number of rules, average number of conditions for a rule, and the accuracy. The effectiveness of the proposed approach is clearly demonstrated by the experimental results on a set of benchmark data mining classification problems.

A rough set-based multiple criteria linear programming approach for the medical diagnosis and prognosis

It is well known that data mining is a process of discovering unknown, hidden information from a large amount of data, extracting valuable information, and using the information to make important business decisions. And data mining has been developed into a new information technology, including regression, decision tree, neural network, fuzzy set, rough set, and support vector machine. This paper puts forward a rough set-based multiple criteria linear programming (RS-MCLP) approach for solving classification problems in data mining. Firstly, we describe the basic theory and models of rough set and multiple criteria linear programming (MCLP) and analyse their characteristics and advantages in practical applications. Secondly, detailed analysis about their deficiencies are provided, respectively. However, because of the existing mutual complementarities between them, we put forward and build the RS-MCLP methods and models which sufficiently integrate their virtues and overcome the adverse factors simultaneously. In addition, we also develop and implement these algorithm and models in SAS and Windows system platforms. Finally, many experiments show that the RS-MCLP approach is prior to single MCLP model and other traditional classification methods in data mining, and remarkably improve the accuracy of medical diagnosis and prognosis simultaneously.

Ensemble of Evolving Neural Networks in Classification

The ensemble of evolving neural networks, which employs neural networks and genetic algorithms, is developed for classification problems in data mining. This network meets data mining requirements such as smart architecture, user interaction, and performance. The evolving neural network has a smart architecture in that it is able to select inputs from the environment and controls its topology. A built-in objective function of the network offers user interaction for customized classification. The bagging technique, which uses a portion of the training set in multiple networks, is applied to the ensemble of evolving neural networks in order to improve classification performance. The ensemble of evolving neural networks is tested by various data sets and produces better performance than both classical neural networks and simple ensemble methods.