Visual learning by integrating descriptive and generative methods

Abstract

This paper presents a mathematical framework for visual learning that integrates two popular statistical learning paradigms in the literature: (I). Descriptive learning, such as Markov random fields and minimax entropy learning, and (II). Generative learning, such as PCA, ICA, TCA, image coding and HMM. We apply this integrated learning framework to modeling, and we assume that an observed texture image is generated by multiple layers of hidden stochastic texton with each being a window function, like a mini-template or a wavelet, under affine transformations. The spatial arrangements of the textons are characterized by minimax entropy models. The processes generate images by occlusion or linear addition. Thus given a raw input image, the learning framework achieves four goals: (i). Computing the appearance of the textons. (ii) Inferring the hidden stochastic processes. (iii). Learning Gibbs models for each process and (iv). Verifying the learnt textons and Gibbs models through random sampling and texture synthesis. The integrated framework subsumes the minimax entropy learning paradigm and creates a richer class of probability models for visual patterns, which are suited for middle level vision representations. Furthermore we show that the integration of description and generative methods yields a natural and general framework of visual learning. We demonstrate the proposed framework and algorithms on many real images.