Graph-based methods for unsupervised and semi-supervised learning

Abstract

Summary form only given. In many applications of machine learning, it is cheap to collect unlabeled examples but expensive to collect labeled ones. Often, the unlabeled examples can provide useful auxiliary information by revealing that the data has a simple underlying structure (M. Belkin and P. Niyogi, 2004). Of particular interest is the case when seemingly high dimensional data can be described in terms of a small number of degrees of freedom. How can we search for low dimensional structure in high dimensional data (C. J. C. Burges, 2005)? If the data is mainly confined to a low dimensional subspace, then simple linear methods can be used to discover the subspace and estimate its dimensionality. More generally, though, if the data lies on (or near) a low dimensional manifold, then its structure may be highly nonlinear, and linear methods are bound to fail. The talk would have four aims: (i) to provide an overview of graph-based spectral methods for nonlinear dimensionality reduction; (ii) to illustrate how unlabeled examples can be used, in conjunction with labeled examples, to improve classification when the data lies on or near on a low dimensional manifold; (iii) to suggest how low dimensional manifolds may arise from particular types of acoustic feature extraction; (iv) to speculate how recent methods in unsupervised and semi-supervised learning could be generalized to problems involving sequences, as occur in speech and language processing