How to explore the patch space

Abstract

Patches are small images (typically $8\times 8$ to $12\times 12$) extracted from natural images. The ``patch space'' is the set of all observable patches extracted from digital images in the world. This observable space is huge and should permit a sophisticated statistical analysis. In the past ten years, statistical inquiries and applications involving the ``patch space'' have tried to explore its structure on several levels. The first attempts have invalidated models based on PCA or Fourier analysis. Redundant bases (or patch dictionaries) obtained by independent component analysis (ICA) or related processes have tried to find a reduced set of patches on which every other patch obtains a sparse decomposition. Optimization algorithms such as EM have been used to explore the patch space as a Gaussian mixture. The goal of the present paper is to review this literature, and to extend its methodology to gain more insight on the independent components of the patch space.  &nbsp The conclusion of our analysis is that the sophisticated ICA tools introduced to analyze the patch space require a previous geometric normalization step to yield non trivial results. Indeed, we demonstrate by a simple experimental setup and by the analysis of the literature that, without this normalization, the patch space structure is actually hidden by the rotations, translations, and contrast changes. Thus, ICA models applied on a random set of patches boil down to segmenting the patch space depending on insignificant dimensions such as the patch orientation or the position of its gradient barycenter. When, instead of exploring the raw patches, one decides to explore the quotient of the set of patches by these action groups, a geometrically interpretable patch structure is revealed.