Cluster-based Classification Research Articles

Recognition of Hand Written Kannada Numerals Using K-Medoids

Data Clustering is considered as an interesting approach for finding similarities in data and putting similar data into groups. Clustering partitions a data set into several groups such that the similarity within a group is larger than that among groups. This paper explores the cluster-based classification scheme in the context of recognition of handwritten Kannada numerals. In this paper, K-Medoids clustering algorithm is being used for the classification. The features used for the classification are obtained from the directional chain code information of the contour points of the numerals. The proposed algorithm is experimented on nearly 1000 samples of handwritten Kannada numerals and obtained 97% of recognition accuracy.

MULTICLASS CLASSIFICATION BASED ON META PROBABILITY CODES

This paper proposes a new approach to improve multiclass classification performance by employing Stacked Generalization structure and One-Against-One decomposition strategy. The proposed approach encodes the outputs of all pairwise classifiers by implicitly embedding two-class discriminative information in a probabilistic manner. The encoded outputs, called Meta Probability Codes (MPCs), are interpreted as the projections of the original features. It is observed that MPC, compared to the original features, has more appropriate features for clustering. Based on MPC, we introduce a cluster-based multiclass classification algorithm, called MPC-Clustering. The MPC-Clustering algorithm uses the proposed approach to project an original feature space to MPC, and then it employs a clustering scheme to cluster MPCs. Subsequently, it trains individual multiclass classifiers on the produced clusters to complete the procedure of multiclass classifier induction. The performance of the proposed algorithm is extensively evaluated on 20 datasets from the UCI machine learning database repository. The results imply that MPC-Clustering is quite efficient with an improvement of 2.4% overall classification rate compared to the state-of-the-art multiclass classifiers.

Robust formulations for clustering-based large-scale classification

Chebyshev-inequality-based convex relaxations of Chance-Constrained Programs (CCPs) are shown to be useful for learning classifiers on massive datasets. In particular, an algorithm that integrates efficient clustering procedures and CCP approaches for computing classifiers on large datasets is proposed. The key idea is to identify high density regions or clusters from individual class conditional densities and then use a CCP formulation to learn a classifier on the clusters. The CCP formulation ensures that most of the data points in a cluster are correctly classified by employing a Chebyshev-inequality-based convex relaxation. This relaxation is heavily dependent on the second-order statistics. However, this formulation and in general such relaxations that depend on the second-order moments are susceptible to moment estimation errors. One of the contributions of the paper is to propose several formulations that are robust to such errors. In particular a generic way of making such formulations robust to moment estimation errors is illustrated using two novel confidence sets. An important contribution is to show that when either of the confidence sets is employed, for the special case of a spherical normal distribution of clusters, the robust variant of the formulation can be posed as a second-order cone program. Empirical results show that the robust formulations achieve accuracies comparable to that with true moments, even when moment estimates are erroneous. Results also illustrate the benefits of employing the proposed methodology for robust classification of large-scale datasets.

Operative assessment of predicted generalization errors on non-stationary distributions in data-intensive applications

Data-intensive applications use empirical methods to extract consistent information from huge samples. When applied to classification tasks, their aim is to optimize accuracy on unseen data hence a reliable prediction of the generalization error is of paramount importance. Theoretical models, such as Statistical Learning Theory, and empirical estimations, such as cross-validation, can both fit data-mining classification domains very well, provided some crucial assumptions are verified in advance. In particular, the stationary distribution of the observed data is critical, although it is sometimes overlooked in practice. The paper formulates an operative criterion to verify the stationary assumption; the method applies to both theoretical and practical predictions of generalization errors. The analysis addresses the specific case of clustering-based classifiers; the K-Winner Machine (KWM) model is used as a reference for its known theoretical bounds; cross-validation provides an empirical counterpart for practical comparison. The criterion, based on efficient unsupervised clustering-based probability distribution estimation, is tested experimentally on a set of different, data-intensive applications, including: intrusion detection for computer-network security, optical character recognition, text mining and pedestrian detection. Experimental results confirm the effectiveness of the proposed approach to efficiently detect non stationarity.

A linguistic approach to classification of bacterial genomes

In the present paper, 188 prokaryote genomes are classified by separately calculating the compositional spectra for the coding and the non-coding parts of the genomes. For each subsequence, the compositional spectrum is transformed into the corresponding point in a vector space. This enables the categorization of genomes into meaningful groups by a formal method. Repeated clustering performed for the coding and the non-coding genome parts makes it possible to estimate the true number of the genome clusters. The method we propose is based on a new application of external cluster validation indexes and on the misclassified quantities obtained in the process of repeated clustering. Besides, we have constructed additional data embedding into the appropriate Euclidean space only on the basis of the distances between compositional spectra. Biological evaluation of the results obtained for the 4-letter and the 2-letter alphabets substantiates the appropriateness of the resulting cluster-based classification.

On-line evolving image classifiers and their application to surface inspection

In this paper, we present image classifiers which are able to adapt and evolve themselves at an on-line machine vision system. These classifiers are initially trained on some pre-labelled training data and further updated based on newly recorded samples, for instance during a production process. The evolution and adaptation mechanism is necessary in order to guarantee a process-save on-line system as usually the pre-labelled data does not cover all possible operating conditions, system states or image classes. It is also recommended for a refinement of the classifiers during the on-line mode in order to boost predictive performance with more loaded samples. We will present two types of on-line evolving image classifiers: The first one is a clustering-based classification approach, which exploits conventional vector quantization, forming an incremental evolving variant around it and extending it to the supervised classification case. The second one is an evolving fuzzy classifier approach which comes with two model architectures, classical single model and a novel multi-model architecture, the later exploiting indicator matrices/vectors for training. The approaches are evaluated in three different on-line surface inspection systems dealing with CD imprint inspection, egg inspection and inspection of metal rotor parts. The evaluation will show the impact of on-line evolved versus ‘static’ classifiers kept fixed during the whole on-line process.

A fuzzy decision tree method for fault classification in the steam generator of a pressurized water reactor

This paper extends a method previously introduced by the authors for building a transparent fault classification algorithm by combining the fuzzy clustering, fuzzy logic and decision trees techniques. The baseline method transforms an opaque, fuzzy clustering-based classification model into a fuzzy logic inference model based on linguistic rules which can be represented by a decision tree formalism. The classification model thereby obtained is transparent in that it allows direct interpretation and inspection of the model. An extension in the procedure for the development of the fuzzy logic inference model is introduced to allow the treatment of more complicated cases, e.g. splitted and overlapping clusters. The corresponding computational tool developed relies on a number of parameters which can be tuned by the user to optimally compromise the level of transparency of the classification process and its efficiency. A numerical application is presented with regards to the fault classification in the Steam Generator of a Pressurized Water Reactor.

Cluster-based classification using self-organising maps for medical image databases

Images are a fundamental source of information in medicine. They can support doctors and students in diagnostic decisions besides providing research and didactic material. The images stored in a database and divided in categories are an important step for data mining and content-based image retrieval (CBIR). This work addresses a methodology which joins the use of discrete wavelet transforms to characterise images and self-organising maps (SOM) neural networks to cluster based classification of medical images. This data mining methodology can be used in categorisation and in computer-aided diagnostic decision.

Development and validation of a cluster-based classification system to facilitate treatment tailoring

The objectives of this study were to replicate smoker profiles identified in Batra et al. (in press) and to develop a cluster-based classification system to categorize new cases into smoker profiles so that an appropriate tailored intervention could be applied. Participants were smokers in southwest Germany who sought treatment for smoking cessation. In the first sample, discriminant analysis was used to create classification formulas for a future study (classification sample: n = 165). The second sample served to replicate the smoker profiles, which included participants reporting symptoms of hyperactivity/novelty-seeking, depressivity or high nicotine dependence as well as participants scoring low across smoking and psychological variables (replication sample: N = 134). Part 1 was focused on the development of formulas, using Fisher's coefficients, with which new cases could be classified. Part 2 adequately replicated previous findings concerning the smoker profiles, such that 70% of participants in the second sample were classified identically using cluster analysis and classification formulas. The smoker profiles found in a previous study were replicated, and a classification system was developed for a future study which will test the efficacy of tailored treatments for the different smoker profiles.

Consensus Clustering of Gene Expression Data and its Application to Gene Function Prediction

Predicting functions of genes is an important issue in biology. Clustering gene expression profiles has been widely used for gene function prediction, but most clustering methods are unstable and sensitive to input parameters such as starting values and number of clusters. In this article, we develop a novel consensus clustering method to address the instability issue and thus improve the performance of clustering methods. The biological function of an unannotated gene is predicted based on the most enriched functional category in its consensus cluster. The MIPS gene annotations are used to evaluate the predictive performance. It is shown that the consensus clustering-based classification method has a significantly better predictive performance than a previously used clustering-based classification method while performing as well as support vector machines (SVMs). In addition to the obvious applicability of consensus clustering to unsupervised learning, the method's advantages in supervised learning include its being a multiclass classifier that can be trained much faster than SVMs, its generality to include any of the many existing clustering algorithms, and its flexibility to be integrated with other predictive models built with other types of data, suggesting its potential for further improved performance. As a concrete example, we consider its combined use with protein-protein interaction data for gene function prediction. It is shown that the combined analysis has a significantly higher predictive accuracy and a much broader functional coverage than using either data source alone.

Squeezing the last drop: Cluster-based classification algorithm

In this paper we propose a new approach for classification problems and apply it to eight problems. A classification problem with a large feature set is partitioned using clustering on a subset of the features. A multi-label classifier is then trained individually on each cluster, using automatic feature selection to customize the feature set for the cluster. The algorithm achieves one to two percent higher accuracy for most of the problems investigated in this study.

An incremental cluster-based approach to spam filtering

As email becomes a popular means for communication over the Internet, the problem of receiving unsolicited and undesired emails, called spam or junk mails, severely arises. To filter spam from legitimate emails, automatic classification approaches using text mining techniques are proposed. This kind of approaches, however, often suffers from low recall rate due to the natures of spam, skewed class distributions and concept drift. This research is thus to propose an appropriate classification approach to alleviating the problems of skewed class distributions and drifting concepts. A cluster-based classification method, called ICBC, is developed accordingly. ICBC contains two phases. In the first phase, it clusters emails in each given class into several groups, and an equal number of features (keywords) are extracted from each group to manifest the features in the minority class. In the second phase, we capacitate ICBC with an incremental learning mechanism that can adapt itself to accommodate the changes of the environment in a fast and low-cost manner. Three experiments are conducted to evaluate the performance of ICBC. The results show that ICBC can effectively deal with the issues of skewed and changing class distributions, and its incremental learning can also reduce the cost of re-training. The feasibility of the proposed approach is thus justified.

Contextual clustering for configuring collaborative conservation of watersheds in the Mid-Atlantic Highlands

The environmental purpose is to characterize watersheds in a region regarding vulnerability and resiliency relative to present and potential degradation of water quality due to human impact based on available spatial information and multidisciplinary expertise. Available information is of six general types as (1) physical and topographic conformation, (2) soil factors, (3) climatic factors, (4) hydrologic characteristics, (5) land-cover/land-use, and (6) prior records of sampling at selected locations for water quality and biological indicators. The strategy is first to develop cluster-based classes of watersheds that are expected to have similar responses to anthropogenic stressors, without using indicators of landscape condition that are directly influenced by local human activity. Watersheds in these classes can then be analyzed for degree of human influence as indicated by land-cover/land-use demographics. More sparse data on water quality and biological indicators at stream sampling locations provide a basis for determining the degradation response to human-induced stressors in each class along with potential for remediation. Focus in this paper is on the first task of cluster-based classification. Statistical adaptation comes in combining empirical objectivity of clustering with interdisciplinary environmental expertise, such that the trajectory of investigation arises from team expertise while the formulation is shaped statistically. Expertise enters initially in recognizing subsets of available descriptors that characterize different aspects of the watershed context needing to be explored separately rather than being completely confounded. Reduction of redundancy among available descriptors and removal of outliers are preliminary concerns. Clustering then proceeds through a series of phases using the sets of variables individually and in selected combinations. Contingency of composite clustering relative to separately clustered sets is examined via special cross tabulations in order to elucidate interactions between sets of variables. The spatial nature of the investigation contributes the major contextual capability for exercising team expertise through visualization using geographic information systems (GIS) that enhances and integrates insights from clustering, particularly with regard to spatial distribution of cluster membership.

Tissue classification with gene expression profiles.

Constantly improving gene expression profiling technologies are expected to provide understanding and insight into cancer-related cellular processes. Gene expression data is also expected to significantly aid in the development of efficient cancer diagnosis and classification platforms. In this work we examine three sets of gene expression data measured across sets of tumor(s) and normal clinical samples: The first set consists of 2,000 genes, measured in 62 epithelial colon samples (Alon et al., 1999). The second consists of approximately equal to 100,000 clones, measured in 32 ovarian samples (unpublished extension of data set described in Schummer et al. (1999)). The third set consists of approximately equal to 7,100 genes, measured in 72 bone marrow and peripheral blood samples (Golub et al, 1999). We examine the use of scoring methods, measuring separation of tissue type (e.g., tumors from normals) using individual gene expression levels. These are then coupled with high-dimensional classification methods to assess the classification power of complete expression profiles. We present results of performing leave-one-out cross validation (LOOCV) experiments on the three data sets, employing nearest neighbor classifier, SVM (Cortes and Vapnik, 1995), AdaBoost (Freund and Schapire, 1997) and a novel clustering-based classification technique. As tumor samples can differ from normal samples in their cell-type composition, we also perform LOOCV experiments using appropriately modified sets of genes, attempting to eliminate the resulting bias. We demonstrate success rate of at least 90% in tumor versus normal classification, using sets of selected genes, with, as well as without, cellular-contamination-related members. These results are insensitive to the exact selection mechanism, over a certain range.

SOORLS: A SOFTWARE REUSE APPROACH ON THE WEB

The Summary-based Object-Oriented Reuse Library System (SOORLS) was developed to support both librarians who manage databases of object-oriented reusable components, and software developers who intend to use these components to develop software on the Web. This paper presents the library management functions implemented by SOORLS, with focus on a software reuse approach based on the summary contents of the library. The cluster-based classification scheme proposed in this paper alleviates the labor-intensity domain analysis problem often attributed to traditional facet-based classification schemes. We then concentrate on the facilities offered by SOORLS' tools, as well as its Web-based architecture, which allows distributed access to reusable components on servers from a variety of platforms.

Coarse classification of Chinese characters via stroke clustering method

In this paper, we propose a stroke clustering-based coarse classification mechanism to classify the multi-fonts Chinese characters. The main purpose of the proposed method is to identify the associating type of an input character together with the extraction of its embedded composing components. In this paper, the K-mean clustering algorithm is employed to cluster the thinned strokes. Besides, mis-clustered stroke modification techniques are developed to rearrange the mis-clustered strokes generated by the K-mean algorithm. Five kinds of fonts for 2500 frequently used Chinese characters are tested in our experiments. The average classification rate is 92.57% which is very promising for coarse classification.

An examination of cluster-based classification schemes for DUI offenders.

This study examines the utility of cluster-based classification schemes for DUI offenders. Variables from previous empirical typologies and multiple domains were used in a series of cluster analyses in order to examine replicability across independent samples, across clustering algorithms and across sets of psychometric indicators. The arbitrary nature of cluster solutions and the external validity of cluster-based schemes, with respect to an outcome criterion constructed synthetically from earlier research, was examined. Only three of eight cluster techniques (Ward's, K means, and complete linkage) yielded meaningful results. For these techniques, replicability was poor across samples, algorithms and sets of psychometric indicators. Results indicated that identified clusters were arbitrary. Although external validity analysis yielded positive results, the possibility that external validity was a function of underlying dimensions was discussed, as were other implications of the findings.