Representing and recognizing objects with massive local image patches

Abstract

Natural image patches are fundamental elements for visual pattern modeling and recognition. By studying the intrinsic manifold structures in the space of image patches, this paper proposes an approach for representing and recognizing objects with a massive number of local image patches (e.g. 17x17 pixels). Given a large collection (>10^4) of proto image patches extracted from objects, we map them into two types of manifolds with different metrics: explicit manifolds of low dimensions for structural primitives, and implicit manifolds of high dimensions for stochastic textures. We define these manifolds grown from patches as the ''@e-balls'', where @e corresponds to the perception residual or fluctuation. Using these @e-balls as features, we present a novel generative learning algorithm by the information projection principle. This algorithm greedily stepwise pursues the object models by selecting sparse and independent @e-balls (say 10^3 for each category). During the detection and classification phase, only a small number (say 20) of features are activated by a fast KD-tree indexing technique. The proposed method owns two characters. (1) Automatically generating features (@e-balls) from local image patches rather than designing marginal feature carefully and category-specifically. (2) Unlike the weak classifiers in the boosting models, these selected @e-ball features are used to explain object in a generative way and are mutually independent. The advantage and performance of our approach is evaluated on several challenging datasets with the task of localizing objects against appearance variance, occlusion and background clutter.