Dictionary learning in the analysis sparse representation with optimization on Stiefel manifold

Abstract

Sparse representation has been proven to be a powerful tool for signals and images processing. This paper addresses sparse representation with the so-called analysis model. We pose the problem as to learn an analysis dictionary from signals using an optimization formulation with an orthogonal constraint. The conventional ways for the dictionary update with the orthogonal constraint are first just update in the embedding Euclidean space and then project the result to the manifold on which the constraint is satisfied. Such a manifold is termed as the Stiefel manifold in the literature if the constraint is about orthogonality of the dictionary. However, such a method is an approximate and may not capture the intrinsic structure of the inherent dictionary. How to solve such a problem in a more precise method and to work out a more effective algorithm is the purpose of this paper. Therefore, we propose a novel optimization technique to learn the dictionary along the manifold seamlessly. Numerical experiments on recovery of analysis dictionary show the effectiveness of the proposed algorithm. In addition, for realistic applications, the proposed algorithms show good performances in signal denoising and classification.