Online Coregularization for Multiview Semisupervised Learning

Boliang Sun,Li Jia,Kuihua Huang,Guohui Li

doi:10.1155/2013/398146

Abstract

We propose a novel online coregularization framework for multiview semisupervised learning based on the notion of duality in constrained optimization. Using the weak duality theorem, we reduce the online coregularization to the task of increasing the dual function. We demonstrate that the existing online coregularization algorithms in previous work can be viewed as an approximation of our dual ascending process using gradient ascent. New algorithms are derived based on the idea of ascending the dual function more aggressively. For practical purpose, we also propose two sparse approximation approaches for kernel representation to reduce the computational complexity. Experiments show that our derived online coregularization algorithms achieve risk and accuracy comparable to offline algorithms while consuming less time and memory. Specially, our online coregularization algorithms are able to deal with concept drift and maintain a much smaller error rate. This paper paves a way to the design and analysis of online coregularization algorithms.

Highlights

Semi-supervised learning (S2L) is a relatively new subfield of machine learning which has become a popular research topic throughout the last two decades [1,2,3,4,5,6]
In this paper we presented an online coregularization framework based on the notion of ascending the dual function
We demonstrated that the existing online coregularization algorithms in previous work can be viewed as an approximation of our dual ascending process using gradient ascent

Summary

Introduction

Semi-supervised learning (S2L) is a relatively new subfield of machine learning which has become a popular research topic throughout the last two decades [1,2,3,4,5,6]. Different from standard supervised learning (SL), the S2L paradigm learns from both labeled and unlabeled examples. We investigate the online semi-supervised learning (OS2L) problems with multiple views which have four features: (1) data is abundant, but the resources to label them are limited; (2) data arrives in a stream and cannot store them all; (3) the target functions in each view agree on labels of most examples (compatibility assumption); (4) the views are independent given the labels (independence assumption). The learner is given a training example and is required to predict the label if the example is unlabeled. The challenge of OS2L is that we do not observe the true label for unlabeled examples to evaluate the performance of prediction mechanism. If we want to update the prediction mechanism, we have to rely on indirect forms of feedback

Objectives

Methods

Conclusion