A General Theory of Singular Values with Applications to Signal Denoising

Abstract

We study the Pareto frontier for two competing norms $\|\cdot\|_X$ \nomenclature[X]$\lVert\cdot\rVert_X,\lVert\cdot\rVert_Y$norms and $\|\cdot\|_Y$ on a vector space. For a given vector $c$, the Pareto frontier describes the possible values of $(\|a\|_X,\|b\|_Y)$ for a decomposition $c=a+b$. The singular value decomposition of a matrix is closely related to the Pareto frontier for the spectral and nuclear norm. We will develop a general theory that extends the notion of singular values of a matrix to arbitrary finite dimensional Euclidean vector spaces equipped with dual norms. This also generalizes the diagonal singular value decompositions (DSVDs) for tensors introduced by the author in previous work. We can apply the results to denoising, where $c$ is a noisy signal, $a$ is a sparse signal, and $b$ is noise. Applications include 1D total variation denoising, 2D total variation Rudin--Osher--Fatemi image denoising, LASSO, basis pursuit denoising, and tensor decompositions.