Representation Learning Techniques: An Overview

Abstract

Representation learning techniques, as a paradigm shift in feature generation, are considered as an important and inevitable part of state of the art pattern recognition systems. These techniques attempt to extract and abstract key information from raw input data. Representation learning based methods of feature generation are in contrast to handy feature generation methods which are mainly based on the prior knowledge of expert about the task at hand. Moreover, new techniques of representation learning revolutionized modern pattern recognition systems. Representation learning methods are considered in four main approaches: sub-space based, manifold based, shallow architectures, and deep architectures. This study demonstrates deep architectures are considered as one of the most important methods of representation learning as they cover more general priors of real-world intelligence as a necessity for modern intelligent systems. In other words, deep architectures overcome limitations of their shallow counterparts. In this study, the relationships between various representation learning techniques are highlighted and their advantages and disadvantages are discussed.