Thousands Of Classes Research Articles

Sparse Output Coding for Scalable Visual Recognition

Many vision tasks require a multi-class classifier to discriminate multiple categories, on the order of hundreds or thousands. In this paper, we propose sparse output coding, a principled way for large-scale multi-class classification, by turning high-cardinality multi-class categorization into a bit-by-bit decoding problem. Specifically, sparse output coding is composed of two steps: efficient coding matrix learning with scalability to thousands of classes, and probabilistic decoding. Empirical results on object recognition and scene classification demonstrate the effectiveness of our proposed approach.

Hierarchical Bayesian Inference and Recursive Regularization for Large-Scale Classification

In this article, we address open challenges in large-scale classification, focusing on how to effectively leverage the dependency structures (hierarchical or graphical) among class labels, and how to make the inference scalable in jointly optimizing all model parameters. We propose two main approaches, namely the hierarchical Bayesian inference framework and the recursive regularization scheme. The key idea in both approaches is to reinforce the similarity among parameter across the nodes in a hierarchy or network based on the proximity and connectivity of the nodes. For scalability, we develop hierarchical variational inference algorithms and fast dual coordinate descent training procedures with parallelization. In our experiments for classification problems with hundreds of thousands of classes and millions of training instances with terabytes of parameters, the proposed methods show consistent and statistically significant improvements over other competing approaches, and the best results on multiple benchmark datasets for large-scale classification.

A Meta-Top-Down Method for Large-Scale Hierarchical Classification

Recent large-scale hierarchical classification tasks typically have tens of thousands of classes on which the most widely used approach to multiclass classification--one-versus-rest--becomes intractable due to computational complexity. The top-down methods are usually adopted instead, but they are less accurate because of the so-called error-propagation problem in their classifying phase. To address this problem, this paper proposes a meta-top-down method that employs metaclassification to enhance the normal top-down classifying procedure. The proposed method is first analyzed theoretically on complexity and accuracy, and then applied to five real-world large-scale data sets. The experimental results indicate that the classification accuracy is largely improved, while the increased time costs are smaller than most of the existing approaches.

Research on Java Development Kit Based on Complex Networks

Java is a computer programming language that is class-based, object-oriented, java is, as of 2013, one of the most popular programming languages in use. The Java Development Kit (JDK) is an implementation of java language, which consists of a Java Virtual Machine and all of the java class libraries. There are thousands of classes in the java class libraries, and there are many kinds of relationships between classes. In this paper, we use complex network theory to study the topology of the java class libraries, according to analyzing to the networks feature characters, important classes and interfaces in java class libraries are found, we hope to find the relationship between the characters of complex networks and software reliability.

Multiclass from binary: expanding one-versus-all, one-versus-one and ECOC-based approaches.

Recently, there has been a lot of success in the development of effective binary classifiers. Although many statistical classification techniques have natural multiclass extensions, some, such as the support vector machines, do not. The existing techniques for mapping multiclass problems onto a set of simpler binary classification problems run into serious efficiency problems when there are hundreds or even thousands of classes, and these are the scenarios where this paper's contributions shine. We introduce the concept of correlation and joint probability of base binary learners. We learn these properties during the training stage, group the binary leaner's based on their independence and, with a Bayesian approach, combine the results to predict the class of a new instance. Finally, we also discuss two additional strategies: one to reduce the number of required base learners in the multiclass classification, and another to find new base learners that might best complement the existing set. We use these two new procedures iteratively to complement the initial solution and improve the overall performance. This paper has two goals: finding the most discriminative binary classifiers to solve a multiclass problem and keeping up the efficiency, i.e., small number of base learners. We validate and compare the method with a diverse set of methods of the literature in several public available datasets that range from small (10 to 26 classes) to large multiclass problems (1000 classes) always using simple reproducible scenarios.

Parallel multiclass stochastic gradient descent algorithms for classifying million images with very-high-dimensional signatures into thousands classes

The new parallel multiclass stochastic gradient descent algorithms aim at classifying million images with very-high-dimensional signatures into thousands of classes. We extend the stochastic gradient descent (SGD) for support vector machines (SVM-SGD) in several ways to develop the new multiclass SVM-SGD for efficiently classifying large image datasets into many classes. We propose (1) a balanced training algorithm for learning binary SVM-SGD classifiers, and (2) a parallel training process of classifiers with several multi-core computers/grid. The evaluation on 1000 classes of ImageNet, ILSVRC 2010 shows that our algorithm is 270 times faster than the state-of-the-art linear classifier LIBLINEAR.

Good Practice in Large-Scale Learning for Image Classification

We benchmark several SVM objective functions for large-scale image classification. We consider one-versus-rest, multiclass, ranking, and weighted approximate ranking SVMs. A comparison of online and batch methods for optimizing the objectives shows that online methods perform as well as batch methods in terms of classification accuracy, but with a significant gain in training speed. Using stochastic gradient descent, we can scale the training to millions of images and thousands of classes. Our experimental evaluation shows that ranking-based algorithms do not outperform the one-versus-rest strategy when a large number of training examples are used. Furthermore, the gap in accuracy between the different algorithms shrinks as the dimension of the features increases. We also show that learning through cross-validation the optimal rebalancing of positive and negative examples can result in a significant improvement for the one-versus-rest strategy. Finally, early stopping can be used as an effective regularization strategy when training with online algorithms. Following these "good practices," we were able to improve the state of the art on a large subset of 10K classes and 9M images of ImageNet from 16.7 percent Top-1 accuracy to 19.1 percent.

Maxi-Min discriminant analysis via online learning

Linear Discriminant Analysis (LDA) is an important dimensionality reduction algorithm, but its performance is usually limited on multi-class data. Such limitation is incurred by the fact that LDA actually maximizes the average divergence among classes, whereby similar classes with smaller divergence tend to be merged in the subspace. To address this problem, we propose a novel dimensionality reduction method called Maxi-Min Discriminant Analysis (MMDA). In contrast to the traditional LDA, MMDA attempts to find a low-dimensional subspace by maximizing the minimal (worst-case) divergence among classes. This "minimal" setting overcomes the problem of LDA that tends to merge similar classes with smaller divergence when used for multi-class data. We formulate MMDA as a convex problem and further as a large-margin learning problem. One key contribution is that we design an efficient online learning algorithm to solve the involved problem, making the proposed method applicable to large scale data. Experimental results on various datasets demonstrate the efficiency and the efficacy of our proposed method against five other competitive approaches, and the scalability to the data with thousands of classes.

Cross-Validation Optimization for Large Scale Structured Classification Kernel Methods

We propose a highly efficient framework for penalized likelihood kernel methods applied to multi-class models with a large, structured set of classes. As opposed to many previous approaches which try to decompose the fitting problem into many smaller ones, we focus on a Newton optimization of the complete model, making use of model structure and linear conjugate gradients in order to approximate Newton search directions. Crucially, our learning method is based entirely on matrix-vector multiplication primitives with the kernel matrices and their derivatives, allowing straightforward specialization to new kernels, and focusing code optimization efforts to these primitives only. Kernel parameters are learned automatically, by maximizing the cross-validation log likelihood in a gradient-based way, and predictive probabilities are estimated. We demonstrate our approach on large scale text classification tasks with hierarchical structure on thousands of classes, achieving state-of-the-art results in an order of magnitude less time than previous work. Parts of this work appeared in the conference paper Seeger (2007).

An improved handwritten Chinese character recognition system using support vector machine

This paper describes several techniques improving a Chinese character recognition system. Enhanced nonlinear normalization, feature extraction and tuning kernel parameters of support vector machine on a large data set with thousands of classes, contribute to improvement of the overall system performance. The enhanced nonlinear normalization method not only solves the aliasing problem in the original Yamada et al.’s nonlinear normalization method but also avoids the undue stroke distortion in the peripheral region of the normalized image. The support vector machine is for the first time tested on a large data set composed of several million samples and thousands of classes. The recognition system has achieved a high recognition rate of 99.0% on ETL9B, a handwritten Chinese character database.

Fast SVM training algorithm with decomposition on very large data sets

Training a support vector machine on a data set of huge size with thousands of classes is a challenging problem. This paper proposes an efficient algorithm to solve this problem. The key idea is to introduce a parallel optimization step to quickly remove most of the nonsupport vectors, where block diagonal matrices are used to approximate the original kernel matrix so that the original problem can be split into hundreds of subproblems which can be solved more efficiently. In addition, some effective strategies such as kernel caching and efficient computation of kernel matrix are integrated to speed up the training process. Our analysis of the proposed algorithm shows that its time complexity grows linearly with the number of classes and size of the data set. In the experiments, many appealing properties of the proposed algorithm have been investigated and the results show that the proposed algorithm has a much better scaling capability than Libsvm, SVMlight, and SVMTorch. Moreover, the good generalization performances on several large databases have also been achieved.

METASYNTHETIC APPROACH FOR HANDWRITTEN CHINESE CHARACTER RECOGNITION

The metasynthetic approach for solving complicated problems was proposed in 1990.1 And the characteristics of metasynthetic approach can be summarized as human-machine cooperation and integration. Directed by the idea of metasynthesis, the design of two kinds of handwritten Chinese character recognition systems are given in this article. All the designs focus on incorporating human knowledge into multiple classifier combination, which is different from conventional integration. The first one is multi-stage adaptive weighted multiple classifier combination, in which a neural network for coefficient predicting is trained by supervised learning to provide weights suitable for the input pattern. And the second scheme is based on totally parallel combination, in which human intelligence and computer capabilities are combined together through multi-step supervised learning. The experimental results demonstrate substantial improvement in overall performance for handwritten Chinese character recognition with thousands of classes that must be discriminated.

Experimental comparison of coarse-grained concepts in UML, OML, and TOS

Object-oriented approaches offer coarse-grained modeling concepts to structure complex application systems consisting of hundreds or even thousands of classes. This paper analyses such concepts by using a software engineering experiment. Two factors are controlled: Factor A consists of three object-oriented approaches (unified modeling language (UML), OPEN modeling language (OML), and taxonomic objectsystem (TOS)), and factor B consists of two application systems (a database-oriented application and a process-oriented application). The results show that the coarse-grained concepts of the object-oriented approaches OML and TOS are superior to those of UML when modeling a database-oriented application. From the results it can be concluded that the coarse-grained concepts of UML should be improved in further releases. The data were analysed by using advanced statistical methods.

Foundation of the taxonomic object system

Object-oriented methods are lacking semantically compatible concepts for structuring complex applications consisting of hundreds or even thousands of classes. This paper provides the foundation of a so-called taxonomic object system which provides a framework for object-oriented concepts. Based on this taxonomic object system new object-oriented concepts (package class, subsystem class, system class, enterprise class) are developed that allow large applications to be structured bottom-up as well as top-down. The developed new concepts are fully compatible with the fundamental ideas of the object-oriented paradigm (object identity, abstract data, inheritance). These concepts aid the structuring of large applications and are suitable for improving design documentation. The concepts are exemplified by modeling CORBA object services and CORBA common facilities.

Pattern Classification with Compact Distribution Maps

A difficult problem in classification is representing the class-conditional distributions concisely and faithfully. We propose a way of mapping such distributions and its use in constructing a similarity metric. A classifier using this metric can achieve low error rates and useful confidence scores permitting reliable reject behavior. We illustrate the method by an application in a challenging character recognition problem with thousands of classes. For applications to arbitrary domains, we present a method to automatically construct feature transformations that are suitable for such mappings.