An Efficient Orthonormalization-Free Approach for Sparse Dictionary Learning and Dual Principal Component Pursuit.

Xiaoyin Hu,Xin Liu

doi:10.3390/s20113041

Abstract

Sparse dictionary learning (SDL) is a classic representation learning method and has been widely used in data analysis. Recently, the -norm () maximization has been proposed to solve SDL, which reshapes the problem to an optimization problem with orthogonality constraints. In this paper, we first propose an -norm maximization model for solving dual principal component pursuit (DPCP) based on the similarities between DPCP and SDL. Then, we propose a smooth unconstrained exact penalty model and show its equivalence with the -norm maximization model. Based on our penalty model, we develop an efficient first-order algorithm for solving our penalty model (PenNMF) and show its global convergence. Extensive experiments illustrate the high efficiency of PenNMF when compared with the other state-of-the-art algorithms on solving the -norm maximization with orthogonality constraints.

Highlights

We focus on solving the optimization problem with orthogonality constraints: min
We mainly compare the numerical performance of PenNMF with some state-of-the-art algorithms on Sparse dictionary learning (SDL)
We formulate dual principal component pursuit (DPCP) as a special case of them -norm maximization on the Stiefel manifold proposed for SDL

Summary

Introduction

We focus on solving the optimization problem with orthogonality constraints: min. S.t. W W = I p , where W is the variable, Y ∈ Rn× N is a given data matrix, and I p denotes the identity hP P i1 n. N m m with constant matrix in R p× p. The orthogonality constraints W > W = I p in (1) can be expressed as W ∈ Sn,p := {W ∈ Rn× p |W > W = I p }. Sn,p denotes the Stiefel manifold in real matrix space, and we call it the Stiefel manifold for simplicity in the rest of our paper. The sparse dictionary learning (SDL) exploits the low-dimensional features within a set of unlabeled data, and plays an important role in unsupervised representative learning

Methods

Results

Conclusion