Dimensionality Reduction and Sparse Representation

Abstract

Image representation is a fundamental issue in signal processing, pattern recognition, and computer vision. An efficient image representation can lead to the development of effective algorithms for the interpretation of images. Since Marr proposed the fundamental principle of the primary sketch concept of a scene, many image representations have been developed based on this concept. The primary sketch refers to the edges, lines, regions, and others in an image. These are characteristic features, which can be extracted from an image by transforming an image from the pixel-level to a higher level representation for image understanding. Many techniques developed for pattern recognition can be used for image transform and resulted in efficiently representing the characteristic features (or called intrinsic dimension). Dimensionality reduction (DR) and sparse representation (SR) are two representative schemes frequently used in the transform for reducing the dimension of a dataset. These transformations include principle component analysis (PCA), singular value decomposition (SVD), non-negative matrix factorization (NMF), and sparse coding (SC). The study of the mammal brain suggests that only a small number of active neurons encode sensory information at any given point. This finding has led to the rapid development of sparse coding which refers to a small number of nonzero entries in a feature vector. Hence, it is important to represent the sparsity for the dataset by eliminating the data redundancy for applications. The ultimate goal is to have a compact, efficient, and compressed representation of the input data.