Passive-Aggressive Algorithm Research Articles

New one-class classifiers based on the origin separation approach

The model of the one-class support vector machine (νoc-SVM) is based on the “origin separation approach,” i.e., to add a sample at the origin to the training data for the second class and apply a maximum margin separation as known from the classical SVM (C-SVM). This has been proven only for hard margin separation but a clearly defined relation between the νoc-SVM and the C-SVM is not yet existing. In this work, the origin separation approach is analyzed in more detail. The approach reveals to be a more general concept to relate binary and unary (one-class) classifiers. We prove how its application to the ν-SVM, a variant of the C-SVM, directly results in the νoc-SVM. Furthermore, we apply this concept to the C-SVM and other related methods (balanced relative margin machine, regularized Fisher’s discriminant analysis, online passive-aggressive algorithms) to derive entirely new classifiers. This includes variants that can be updated onlinewhich allows the application on large datasets or on systems with very limited resources.

Coactive Learning for Locally Optimal Problem Solving

Coactive learning is an online problem solving setting where the solutions provided by a solver are interactively improved by a domain expert, which in turn drives learning. In this paper we extend the study of coactive learning to problems where obtaining a globally optimal or near-optimal solution may be intractable or where an expert can only be expected to make small, local improvements to a candidate solution. The goal of learning in this new setting is to minimize the cost as measured by the expert effort over time. We first establish theoretical bounds on the average cost of the existing coactive Perceptron algorithm. In addition, we consider new online algorithms that use cost-sensitive and Passive-Aggressive (PA) updates, showing similar or improved theoretical bounds. We provide an empirical evaluation of the learners in various domains, which show that the Perceptron based algorithms are quite effective and that unlike the case for online classification, the PA algorithms do not yield significant performance gains.

Adaptive Aggregating Multiresolution Feature Coding for Image Classification

The Bag of Visual Words (BoW) model is one of the most popular and effective image classification frameworks in the recent literature. The optimal formation of a visual vocabulary remains unclear, and the size of the vocabulary also affects the performance of image classification. Empirically, larger vocabulary leads to higher classification accuracy. However, larger vocabulary needs more memory and intensive computational resources. In this paper, we propose a multiresolution feature coding (MFC) framework via aggregating feature codings obtained from a set of small visual vocabularies with different sizes, where each vocabulary is obtained by a clustering algorithm, and different clustering algorithm discovers different aspect of image features. In MFC, feature codings from different visual vocabularies are aggregated adaptively by a modified Online Passive-Aggressive Algorithm under the histogram intersection kernel, which lead to a closed-form solution. Experiments demonstrate that the proposed method (1) obtains the same if not higher classification accuracy than the BoW model with a large visual vocabulary; and (2) needs much less memory and computational resources.

Online adaptation strategies for statistical machine translation in post-editing scenarios

One of the most promising approaches to machine translation consists in formulating the problem by means of a pattern recognition approach. By doing so, there are some tasks in which online adaptation is needed in order to adapt the system to changing scenarios. In the present work, we perform an exhaustive comparison of four online learning algorithms when combined with two adaptation strategies for the task of online adaptation in statistical machine translation. Two of these algorithms are already well-known in the pattern recognition community, such as the perceptron and passive-aggressive algorithms, but here they are thoroughly analyzed for their applicability in the statistical machine translation task. In addition, we also compare them with two novel methods, i.e., Bayesian predictive adaptation and discriminative ridge regression. In statistical machine translation, the most successful approach is based on a log-linear approximation to a posteriori distribution. According to experimental results, adapting the scaling factors of this log-linear combination of models using discriminative ridge regression or Bayesian predictive adaptation yields the best performance.

Graph-Based Dependency Parsing Using Dynamic Features in Korean

In this paper, we propose a graph-based dependency parsing model using dynamic features. The dynamic features, which are derived from the partial tree during bottom-up chart parsing, play an important role in finding the correct heads. Based on the beneficial aspects of the head-final property of Korean, we suggest a variant of the CYK parsing algorithm with an O(n3) complexity which has the ability to search for the maximum spanning tree (MST) from all projective trees. Compared to Eisner's algorithm, this algorithm is time-efficient in parsing all other head-final languages. We also explain why the graph-based approaches are seldom attempted in parsing morphologically rich languages (MRLs) and suggest a morpheme-level dependency representation as a solution. Finally, we evaluate the proposed method on the KAIST corpus using two discriminative online learning methods (the averaged perceptron algorithm and passive-aggressive algorithm) and show that the trained parser achieves the best reported parsing accuracy.

Large-Margin Training of Dependency Parsers Using Pegasos Algorithm

This letter presents a modified Pegasos algorithm for graph-based dependency parsing. We show that the modified Pegasos algorithm outperforms the margin infused relaxed and passive aggressive algorithms in three languages.