Handling Classification Problems Research Articles

An Efficient Feature Selection Technique using Supervised Fuzzy Information Theory

The Feature Selection is one of the key processes for knowledge acquisition. Some data set is multidimensional and larger in size. When this data set is used for classification it may end with wrong results and it may also occupy more resources especially in terms of time. Most of the features present are redundant and inconsistent and affect the classification. In order to improve the efficiency of classification these redundancy and inconsistency features must be eliminated. The Feature subset contains the minimum number of features that most contribute to accuracy In this paper, present a method for dealing with feature subset selection based on fuzzy Information measures for handling classification problems. First, to construct the membership function of each fuzzy set of a feature. Then, select the feature subset based on the proposed fuzzy Informationy measure focusing on boundary samples. It also presents an experiment result to show the applicability of the proposed method. The performance of the system is evaluated in MATLAB on several benchmark data sets in the UCI machine learning repository.

On principal component analysis for high-dimensional XCSR

XCSR is an accuracy-based learning classifier system which can handle classification problems with real-value features. However, as the number of features increases, a high classification accuracy comes at the cost of more resources: larger population sizes and longer computational running times. In this paper we investigate PCA-XCSR (a sequential application of PCA and XCSR) in three environments with different characteristics: a discrete and imbalanced environment (KDD’99 network intrusion), a continuous and highly symmetric environment (MiniBooNE), and a highly discrete, highly imbalanced environment (Census/Income (KDD)). These experiments show that in the three different environments, PCA-XCSR, in addition to being able to reduce the computational resources and time requirements of XCSR by approximately 50 %, is able to consistently maintain its high accuracy. In addition to that, it reduces the required population size needed by XCSR. Also, we suggest heuristics for selecting the number of principal components to use when using PCA-XCSR.

Fault Diagnosis of Nuclear Power Equipment Based on HMM-SVM and Database Development

This paper mainly introduced the basic theory of Hidden Markov Model (HMM) and Support Vector Machines (SVM). HMM has strong capability of handling dynamic process of time series and the timing pattern classification, particularly for the analysis of non-stationary, poor reproducibility signals. It has good ability to learn and re-learn and high adaptability. SVM has strong generalization ability of small samples, which is suitable for handling classification problems, to a greater extent, reflecting the differences between categories. Based on the advantages and disadvantages between the two models, this paper presented a hybrid model of HMM-SVM. Experiments showed that the HMM-SVM model was more effective and more accurate than the two single separate models. The paper also explored the application of its database system development, which could help the managers to get and handle the data quickly and effectively.

A Hybrid Higher Order Neural Classifier for handling classification problems

In this paper, we propose a novel Hybrid Higher Order Neural Classifier (HHONC) which contains different high-order units. In contrast with conventional fully-connected higher order neural networks (HONN), our proposed method uses fewer learning parameters and allocates the best fitted model in dealing with different datasets by modifying the orders of different high-order units and updating the learning parameters. Structure, model selection and updating the learning parameters of HHONC is introduced and is applied in classification of the Iris data set, the breast cancer data set, the Wine recognition data set, the Glass identification data set, the Balance scale data set, and the Pima diabetes data set. Acquired results are compared with the methods presented in Chen and Shie (2009). It is observed that the fewer features the dataset contains, the more accurate the HHONC performs, however the accuracy of datasets with more features are acceptable. Experimental results show about 3.5% and 0.6% improvements compared to the best accuracy obtained in previously methods for classifying the Pima diabetes and Iris datasets, respectively. In addition, by using a same method for reducing the feature number, it’s shown the proposed method perform more accurate than methods presented in Shie and Chen (2008). In this case, improvements compared to the best acquired accuracy of mentioned methods are about 1.7%, 1.3% and 0.2% in classification of Pima, Iris and Breast cancer datasets, respectively.

Fuzzy classification systems based on fuzzy information gain measures

In this paper, we present a new method for handling classification problems using a new fuzzy information gain measure. Based on the proposed fuzzy information gain measure, we propose an algorithm for constructing membership functions, calculating the class degree of each subset of training instances with respect to each class and calculating the fuzzy entropy of each subset of training instances. Based on the constructed membership function of each fuzzy set of each feature, the obtained class degree of each subset of training instances with respect to each class and the obtained fuzzy entropy of each subset of training instances, we propose an evaluating function for classifying testing instances. The proposed method gets higher average classification accuracy rates than the methods presented in [John, G. H., & Langley, P. (1995). Estimating continuous distributions in Bayesian classifiers. In Proceedings of the 11th conference on uncertainty in artificial intelligence, Montreal, Canada (pp. 338–345); Platt, J. C. (1999). Using analytic QP and sparseness to speed training of support vector machines. In Proceedings of the 13th annual conference on neural information processing systems, Denver, Colorado (pp. 557–563); Quinlan, J. R. (1993). C4.5: Programs for machine learning. San Francisco: Morgan Kaufmann].

FEATURE EXTRACTION BASED ON DIRECT CALCULATION OF MUTUAL INFORMATION

In many pattern recognition problems, it is desirable to reduce the number of input features by extracting important features related to the problems. By focusing on only the problem-relevant features, the dimension of features can be greatly reduced and thereby can result in a better generalization performance with less computational complexity. In this paper, we propose a feature extraction method for handling classification problems. The proposed algorithm is used to search for a set of linear combinations of the original features, whose mutual information with the output class can be maximized. The mutual information between the extracted features and the output class is calculated by using the probability density estimation based on the Parzen window method. A greedy algorithm using the gradient descent method is used to determine the new features. The computational load is proportional to the square of the number of samples. The proposed method was applied to several classification problems, which showed better or comparable performances than the conventional feature extraction methods.

Feature subset selection based on fuzzy entropy measures for handling classification problems

In this paper, we present a new method for dealing with feature subset selection based on fuzzy entropy measures for handling classification problems. First, we discretize numeric features to construct the membership function of each fuzzy set of a feature. Then, we select the feature subset based on the proposed fuzzy entropy measure focusing on boundary samples. The proposed method can select relevant features to get higher average classification accuracy rates than the ones selected by the MIFS method (Battiti, R. in IEEE Trans. Neural Netw. 5(4):537---550, 1994), the FQI method (De, R.K., et al. in Neural Netw. 12(10):1429---1455, 1999), the OFEI method, Dong-and-Kothari's method (Dong, M., Kothari, R. in Pattern Recognit. Lett. 24(9):1215---1225, 2003) and the OFFSS method (Tsang, E.C.C., et al. in IEEE Trans. Fuzzy Syst. 11(2):202---213, 2003).

A Feature Selection Newton Method for Support Vector Machine Classification

A fast Newton method, that suppresses input space features, is proposed for a linear programming formulation of support vector machine classifiers. The proposed stand-alone method can handle classification problems in very high dimensional spaces, such as 28,032 dimensions, and generates a classifier that depends on very few input features, such as 7 out of the original 28,032. The method can also handle problems with a large number of data points and requires no specialized linear programming packages but merely a linear equation solver. For nonlinear kernel classifiers, the method utilizes a minimal number of kernel functions in the classifier that it generates.

Mining the breast cancer pattern using artificial neural networks and multivariate adaptive regression splines

Data mining is a very popular technique and has been widely applied in different areas these days. The artificial neural network has become a very popular alternative in prediction and classification tasks due to its associated memory characteristics and generalization capability. However, the relative importance of potential input variables and the long training process have often been criticized and hence limited its application in handling classification problems. The objective of the proposed study is to explore the performance of data classification by integrating artificial neural networks with the multivariate adaptive regression splines (MARS) approach. The rationale under the analyses is firstly to use MARS in modeling the classification problem, then the obtained significant variables are used as the input variables of the designed neural networks model. To demonstrate the inclusion of the obtained important variables from MARS would improve the classification accuracy of the networks, diagnostic tasks are performed on one fine needle aspiration cytology breast cancer data set. As the results reveal, the proposed integrated approach outperforms the results using discriminant analysis, artificial neural networks and multivariate adaptive regression splines and hence provides an efficient alternative in handling breast cancer diagnostic problems.

A novel evolutionary data mining algorithm with applications to churn prediction

Classification is an important topic in data mining research. Given a set of data records, each of which belongs to one of a number of predefined classes, the classification problem is concerned with the discovery of classification rules that can allow records with unknown class membership to be correctly classified. Many algorithms have been developed to mine large data sets for classification models and they have been shown to be very effective. However, when it comes to determining the likelihood of each classification made, many of them are not designed with such purpose in mind. For this, they are not readily applicable to such problems as churn prediction. For such an application, the goal is not only to predict whether or not a subscriber would switch from one carrier to another, it is also important that the likelihood of the subscriber's doing so be predicted. The reason for this is that a carrier can then choose to provide a special personalized offer and services to those subscribers who are predicted with higher likelihood to churn. Given its importance, we propose a new data mining algorithm, called data mining by evolutionary learning (DMEL), to handle classification problems of which the accuracy of each predictions made has to be estimated. In performing its tasks, DMEL searches through the possible rule space using an evolutionary approach that has the following characteristics: 1) the evolutionary process begins with the generation of an initial set of first-order rules (i.e., rules with one conjunct/condition) using a probabilistic induction technique and based on these rules, rules of higher order (two or more conjuncts) are obtained iteratively; 2) when identifying interesting rules, an objective interestingness measure is used; 3) the fitness of a chromosome is defined in terms of the probability that the attribute values of a record can be correctly determined using the rules it encodes; and 4) the likelihood of predictions (or classifications) made are estimated so that subscribers can be ranked according to their likelihood to churn. Experiments with different data sets showed that DMEL is able to effectively discover interesting classification rules. In particular, it is able to predict churn accurately under different churn rates when applied to real telecom subscriber data.

Finite Newton method for Lagrangian support vector machine classification

An implicit Lagrangian [Math. Programming Ser. B 62 (1993) 277] formulation of a support vector machine classifier that led to a highly effective iterative scheme [J. Machine Learn. Res. 1 (2001) 161] is solved here by a finite Newton method. The proposed method, which is extremely fast and terminates in 6 or 7 iterations, can handle classification problems in very high dimensional spaces, e.g. over 28,000, in a few seconds on a 400 MHz Pentium II machine. The method can also handle problems with large datasets and requires no specialized software other than a commonly available solver for a system of linear equations. Finite termination of the proposed method is established in this work.

GENERATING FUZZY RULES FROM TRAINING DATA CONTAINING NOISE FOR HANDLING CLASSIFICATION PROBLEMS

It is obvious that one of the important tasks in a fuzzy system is to find a set of rules to deal with a specific classification problem. In recent years, many researchers focused on the research topic of generating fuzzy rules from training data for handling classification problems. In a previous paper, we presented an algorithm to construct membership functions and to generate fuzzy rules from training examples. In this paper, we extend that work to propose a new algorithm to generate fuzzy rules from training data containing noise to deal with classification problems. The proposed algorithm gets a higher classification accuracy rate and generates fewer fuzzy rules and fewer input attributes in the antecedent portions of the generated fuzzy rules.

GENERATING FUZZY RULES FROM TRAINING DATA CONTAINING NOISE FOR HANDLING CLASSIFICATION PROBLEMS

It is obvious that one of the important tasks in a fuzzy system is to find a set of rules to deal with a specific classification problem. In recent years, many researchers focused on the research topic of generating fuzzy rules from training data for handling classification problems. In a previous paper, we presented an algorithm to construct membership functions and to generate fuzzy rules from training examples. In this paper, we extend that work to propose a new algorithm to generate fuzzy rules from training data containing noise to deal with classification problems. The proposed algorithm gets a higher classification accuracy rate and generates fewer fuzzy rules and fewer input attributes in the antecedent portions of the generated fuzzy rules.

A new method for generating fuzzy rules from numerical data for handling classification problems

(2001). A new method for generating fuzzy rules from numerical data for handling classification problems. Applied Artificial Intelligence: Vol. 15, No. 7, pp. 645-664.

FUZZY REASONING BY CASE FOR DECISION SUPPORT SYSTEMS

This paper presents a selection of alternative approaches to handle classification problems using the paradigm of case-based reasoning with fuzzy concepts. Our main concern is classification and pattern recognition queries in a fuzzy environment. An example was developed to explain the various methods and the results compared.