Online Manifold Regularization by Dual Ascending Procedure

Bin Sun ,Hui Zhang,Guohui Li,Li Jia

doi:10.1155/2013/838439

Bin Sun , Hui Zhang + Show 2 more

Open Access

https://doi.org/10.1155/2013/838439

Copy DOI

Abstract

We propose a novel online manifold regularization framework based on the notion of duality in constrained optimization. The Fenchel conjugate of hinge functions is a key to transfer manifold regularization from offline to online in this paper. Our algorithms are derived by gradient ascent in the dual function. For practical purpose, we propose two buffering strategies and two sparse approximations to reduce the computational complexity. Detailed experiments verify the utility of our approaches. An important conclusion is that our online MR algorithms can handle the settings where the target hypothesis is not fixed but drifts with the sequence of examples. We also recap and draw connections to earlier works. This paper paves a way to the design and analysis of online manifold regularization algorithms.

Highlights

Semisupervised learning (S2L) of different classifiers is an important problem in machine learning with interesting theoretical properties and practical applications [1,2,3,4,5]
We investigate the online semisupervised learning (OS2L) problems which have three features as follows: (i) data is abundant but the resources to label them are limited; (ii) data arrives in a stream and cannot even store them all; (iii) no statistical assumptions are found, which means that p(x, y) can change over time
We provide some additional results along the online manifold regularization (MR) algorithms run and discuss more precisely the effect of our derived online MR algorithms

Summary

Introduction

Semisupervised learning (S2L) of different classifiers is an important problem in machine learning with interesting theoretical properties and practical applications [1,2,3,4,5]. Different from standard supervised learning (SL), the S2L paradigm learns from both labeled and unlabeled examples. OS2L algorithms take place in a sequence of consecutive rounds. The learner is given a training example and is required to predict the label if the example is unlabeled. The learner uses a prediction mechanism which builds a mapping from the set of examples to the set of labels. The challenge of OS2L is that we do not observe the true label for unlabeled examples to evaluate the performance of prediction mechanism. If we want to update the prediction mechanism, we have to rely on indirect forms of feedback

Objectives

Methods

Results

Conclusion