General Classification Problems Research Articles

Approximation algorithms for classification problems with pairwise relationships

In a traditional classification problem, we wish to assign one of k labels (or classes) to each of n objects , in a way that is consistent with some observed data that we have about the problem. An active line of research in this area is concerned with classification when one has information about pairwise relationships among the objects to be classified; this issue is one of the principal motivations for the framework of Markov random fields, and it arises in areas such as image processing, biometry, and document analysis. In its most basic form, this style of analysis seeks to find a classification that optimizes a combinatorial function consisting of assignment costs ---based on the individual choice of label we make for each object---and separation costs ---based on the pair of choices we make for two "related" objects.We formulate a general classification problem of this type, the metric labeling problem ; we show that it contains as special cases a number of standard classification frameworks, including several arising from the theory of Markov random fields. From the perspective of combinatorial optimization, our problem can be viewed as a substantial generalization of the multiway cut problem, and equivalent to a type of uncapacitated quadratic assignment problem .We provide the first nontrivial polynomial-time approximation algorithms for a general family of classification problems of this type. Our main result is an O (log k log log k )-approximation algorithm for the metric labeling problem, with respect to an arbitrary metric on a set of k labels, and an arbitrary weighted graph of relationships on a set of objects. For the special case in which the labels are endowed with the uniform metric ---all distances are the same---our methods provide a 2-approximation algorithm.

Combination of multiple classifiers for the customer's purchase behavior prediction

In these days, EC companies are eager to learn about their customers using data mining technologies. But the diverse situations of such companies make it difficult to know which is the most effective algorithm for the given problems. Recently, a movement towards combining multiple classifiers has emerged to improve classification results. In this paper, we propose a method for the prediction of the EC customer's purchase behavior by combining multiple classifiers based on genetic algorithm. The method was tested and evaluated using Web data from a leading EC company. We also tested the validity of our approach in general classification problems using handwritten numerals. In both cases, our method shows better performance than individual classifiers and other known combining methods we tried.

Determination of the Prevalence of Infection with Mycobacterium tuberculosis among Persons Vaccinated against Bacillus Calmette-Guerin in South Korea

The prevalence of tuberculous infection was estimated among 12,032 persons with a Bacillus Calmette-Guérin (BCG) vaccination scar and 7,788 persons without such a scar who participated in a nationwide tuberculin skin test survey conducted in the Republic of Korea in 1975. The analysis was built upon mixture models that captured the heterogeneity of indurations arising from tuberculous infection, cross-reactions due to infection with environmental mycobacteria, and BCG vaccination. The three distributions were allowed to vary by age, sex, and BCG vaccination status in the Bayesian manner, according to the prior opinion of the authors. Estimated prevalences of tuberculous infection were similar among persons with a BCG scar and persons without one: 7.5% (95% credibility interval (CI): 3.1, 12.5) and 5.2% (95% CI: 4.2, 6.3), respectively, at age 0-4 years and 87.3% (95% CI: 84.0, 90.2) and 84.0% (95% CI: 81.9, 85.8), respectively, at age 25-29 years. From this analysis it can be concluded that mixture models allow investigators, for the first time, to estimate the prevalence of tuberculous infection not only in unvaccinated persons but also in the BCG-vaccinated population. Mixture models are a versatile tool for analyzing diagnostic test data and more general classification problems of considerable complexity.

Finite type invariants and n-equivalence of 3-manifolds

We announce a new approach to a definition of finite type invariants and introduce a notion of n-equivalence for 3-manifolds with boundary. For the ℤ H S we state equivalence of different definitions of finite type invariants. The theory for 3-manifolds is made completely parallel to the corresponding theory for links. We show how to reduce general classification problems of arbitrary 3-manifolds up to n-equivalences to the study of finite type invariants of string links and integer homology spheres.

A fuzzy logic method for modulation classification in nonideal environments

In this paper, we present a fuzzy logic modulation classifier that works in nonideal environments in which it is difficult or impossible to use precise probabilistic methods. We first transform a general pattern classification problem into one of function approximation, so that fuzzy logic systems (FLS) can be used to construct a classifier; then, we introduce the concepts of fuzzy modulation type and fuzzy decision and develop a nonsingleton fuzzy logic classifier (NSFLC) by using an additive FLS as a core building block. Our NSFLC uses 2D fuzzy sets, whose membership functions are isotropic so that they are well suited for a modulation classifier (MC). We establish that our NSFLC, although completely based on heuristics, reduces to the maximum-likelihood modulation classifier (ML MC) in ideal conditions, In our application of NSFLC to MC in a mixture of /spl alpha/-stable and Gaussian noises, we demonstrate that our NSFLC performs consistently better than the ML MC and it gives the same performance as the ML MC when no impulsive noise is present.

Classification of tetraplegics through automatic movement evaluation

The general problem of classification of functional movements in humans with spinal cord injuries requires the following questions to be answered: what are the essential kinematic parameters that we have to observe during the movement? Is it possible to estimate preserved motor skills based on kinematics? Which computational method for identification is suited to geometric feature analysis? To answer these questions we have developed the methodology which has two phases: (1) recordings of a series of specified arm movements; and (2) custom made software for graphical presentation of arm movements and the design of wavelet and neural networks for movement classification. The proposed protocol is automated and both graphical presentation and neural networks allow easy interpretation of the instrumented assessment to accomplish automatic classification of arm movements in tetraplegics. The protocol was evaluated on 16 spinal cord injury (SCI) patients and seven healthy control subjects for three different arm movements. The classification rate yielded results in the range 46–100% for movement trials that were tested. The application of neural networks for classification of arm movements is completed with results using different neural networks: backpropagation, radial basis, recurent (Elman), self-organizing and Learning Vector Quantization (LVQ).

Classifying rational densities using two one-dimensional cellular automata

Given a (finite but arbitrarily long) string of zeros and ones, we report a way to determine if the number of ones is less than, greater than, or equal to a prescribed number by applying two sets of cellular automaton rules in succession. Thus, we solve the general one-dimensional density classification problem using two cellular automata.

Multigroup Discriminant Analysis Using Linear Programming

In this paper we introduce a nonparametric linear programming formulation for the general multigroup classification problem. Previous research using linear programming formulations has either been limited to the two-group case, or required complicated constraints and many zero-one variables. We develop general properties of our multigroup formulation and illustrate its use with several small example problems and previously published real data sets. A comparative analysis on the real data sets shows that our formulation may offer an interesting robust alternative to parametric statistical formulations for the multigroup discriminant problem.

The Error-Reject Tradeoff

We investigate the error versus reject tradeoff for classifiers. Our analysis is motivated by the remarkable similarity in error-reject tradeoff curves for widely differing algorithms classifying handwritten characters. We present the data in a new scaled version that makes this universal character particularly evident. Based on Chow's theory of the error-reject tradeoff and its underlying Bayesian analysis we argue that such universality is in fact to be expected for general classification problems. Furthermore, we extend Chow's theory to classifiers working from finite samples on a broad, albeit limited, class of problems. The problems we consider are effectively binary, i.e., classification problems for which almost all inputs involve a choice between the right classification and at most one predominant alternative. We show that for such problems at most half of the initially rejected inputs would have been erroneously classified. We show further that such problems arise naturally as small perturbations of the PAC model for large training sets. The perturbed model leads us to conclude that the dominant source of error comes from pairwise overlapping categories. For infinite training sets, the overlap is due to noise and/or poor preprocessing. For finite training sets there is an additional contribution from the inevitable displacement of the decision boundaries due to finiteness of the sample. In either case, a rejection mechanism which rejects inputs in a shell surrounding the decision boundaries leads to a universal form for the error-reject tradeoff. Finally, we analyze a specific reject mechanism based on the extent of consensus among an ensemble of classifiers. For the ensemble reject mechanism we find an analytic expression for the error-reject tradeoff based on a maximum entropy estimate of the problem difficulty distribution.

SEMANTIC CLASSIFICATION OF INSTANCES IN A FRAME-BASED REPRESENTATION

The topic of this paper is classification reasoning in frame-based representations. In these representations, an object designates either a concept (which is represented by a class) or a concrete entity which illustrates one or more classes (represented by an instance). Our objective is to define an instance classification method, which gives a more important place to the semantics of the Object To Classify (OTC) than other works in this area. We have studied cognitive psychology works concerning categorization. These works are focused on the identific:ation of the elements which are concerned with the categorization process. We have taken into account certain results of cognitive psychology to define our classification system. The theme of this paper is the following: in the section 2, the general problem of classification in the frame-based representations is presented. Then, in section 3, critical review of these representations is proposed. The section 4 presents the main results of the cognitive psychology concerning categorization and the section 5 details the more important points of our classification system. Then, the general approach of our system is given (section 6). Finally, we give the elements of knowledge participating into the classification process (section 7) and the matching step is described, specially important in the defined method (section 8).

How to solve the N-bit parity problem with two hidden units

We construct a three-layer feedforward network (one hidden layer) which can solve the N-bit parity problem employing just two hidden units. We discuss the implications of employing problem constraints into transfer functions for general pattern classification problems.

Remarks on pure spinors

General remarks on the significance of spinors are followed by a brief description of spinor connections on low-dimensional spheres and their interpretation as gauge configurations. Cartan's notion of pure spinors is related to the general problem of classification of orbits of the spin group in projective spinor space. There is a nontrivial bundle of pure spinor directions over the conformal compactification of any space with a metric of suitable signature. In higher dimensions, pure spinors introduce natural nonlinearities and lead to topologically nontrivial configurations. It is shown how the constraint defining pure spinors may induce a ‘mass term’ in the Weyl equation for such spinors in a space of signature (3, 4).

4-Manifolds with large symmetry groups

The restriction to non-abelian Lie groups avoids the difficult problem of classifying 4manifolds with S’ or T ‘-actions [3,4,9-l 11. (It is well known that any 4-manifold with an effective T “-action, n 2 3, is T 4 or St x L, where L is a lens space.) This paper thus reduces the general classification problem for closed, orientable 4-manifolds with a compact Lie group G of symmetries to the cases G = S’ or T2. In $1 we show that one need only consider actions of SU (2) or SO (3), and the well known subgroup structure of these two groups is recalled. A complete equivariant classification of SU (2) and SO(3)-actions on 4-manifolds (including the non-orientable case) is given in 9 2. The codimension 1 case is taken largely from the second author’s thesis [ 123. The final section gives the topological classification in the orientable case. This leaves open the non-orientable case, where the situation seems quite interesting and more intricate (cf. [13] for the codimension 1 case). We plan to take this up in a sequel to this paper. We shall work in the smooth category. B” will denote the n-ball, s” the n-sphere, and P” the real projective n-space. The reader is referred to Bredon’s book [l] for the basic definitions and theorems of transformation groups.

Generalized error bounds in pattern recognition

Upper as well as lower bounds on the probability of error for the general pattern classification problem are obtained in terms of generalized entropy. It is proved that the bounds studied here are sharper than several previously known ones. Some properties of generalized entropy are also given.

A martingale approach to the problem of dimensionality and performance of bayesian classifiers: asymptotic results

A martingale approach to the problem of performance of Bayesian classifiers with increasing feature dimensionality is applied here . Martingale limit theorems are also used to demonstrate that the expected probability of correct classification tends monotonically to unity for two general classification problems.

A note on singular lagrangian submanifolds

Classification of singular lagrangian submanifolds which appear as images of a regular one under a symplectic relation, is considered from the point of view of standard singularity theory. The classification is carried out in small dimensions and restricted to special types of symplectic objects. Normal forms for singular pullbacks and pushforwards are given using an appropriate symplectic equivalence group. It is shown that the general classification problem reduces to the classification problem for appropriate mapping diagrams. An approach to the classical theories of phase transition is given based on the geometry of singular lagrangian images. The variational open swallowtails and regularly intersecting pairs of holonomic components are resolved using an appropriate reduction relation. Examples are given of singularities encountered in physics.

On the mixture maximum likelihood approach to estimation and clustering

On the mixture maximum likelihood approach to estimation and clustering

On 4-manifolds with singular torus actions

This paper studies the homeomorphism classification problem for closed oriented 4-manifolds which support effective codimension-two torus actions. With suitable restrictions on the actions a classification is obtained in terms of numerical invariants computable from orbit data. These invariants are related to the fundamental groups and the cohomology rings of the underlying manifolds. They do not distinguish among all 4-manifolds with actions as above, and so the general classification problem remains open.

Multiclass Bayes Error Estimation by a Feature Space Sampling Technique

A general Gaussian M-class N-feature classification problem is defined. An algorithm is developed that requires the class statistics as its only input and computes the minimum probability of error through use of a combined analytical and numerical integration over a sequence simplifying transformations of the feature space. The results are compared with those obtained by conventional techniques applied to a 2-class 4-feature discrimination problem with results previously reported and 4-class 4-feature multispectral scanner Landsat data classified by training and testing of the available data.

Simple Lie algebras of toral rank one

Let L be a finite-dimensional simple Lie algebra over an algebraically closed field of characteristic p > 7 p > 7 . Let L have Cartan decomposition L = H + ∑ γ ∈ Γ L γ L = H + {\sum _{\gamma \in \Gamma }}{L_\gamma } . If Γ \Gamma generates a cyclic group then L is isomorphic to sl ( 2 , F ) {\text {sl}}(2,F) or to one of the simple Lie algebras of generalized Cartan type W ( 1 : n ) W(1:{\mathbf {n}}) or H ( 2 : n : Φ ) ( 2 ) H{(2:{\mathbf {n}}:\Phi )^{(2)}} .