Signal Processing Using Dictionaries, Atoms, and Deep Learning: A Common Analysis-Synthesis Framework

Abstract

Signal decomposition (analysis) and reconstruction (synthesis) are cornerstones in signal processing and feature recognition tasks. Signal decomposition is traditionally achieved by projecting data onto predefined basis functions, often known as atoms. Coefficient manipulation (e.g., thresholding) combined with signal reconstruction then either provides signals with enhanced quality or permits extraction of desired features only. More recently dictionary learning and deep learning have also been actively used for similar tasks. The purpose of dictionary learning is to derive the most appropriate basis functions directly from the observed data. In deep learning, neural networks or other transfer functions are taught to perform either feature classification or data enhancement directly, provided solely some training data. This review shows first how popular signal processing methods, such as basis pursuit and sparse coding, are related to analysis and synthesis. We then explain how dictionary learning and deep learning using neural networks can also be interpreted as generalized analysis and synthesis methods. We introduce the underlying principles of all techniques and then show their inherent strengths and weaknesses using various examples, including two toy examples, a moonscape image, a magnetic resonance image, and geophysical data.