Subspace Approximation with Outliers

Abstract

The subspace approximation problem with outliers, for given n points in d dimensions \(x_{1}, x_{2}, \dotsc , x_{n} \in \mathbb {R}^{d}\), an integer \(1 \le k \le d\), and an outlier parameter \(0 \le \alpha \le 1\), is to find a k-dimensional linear subspace of \(\mathbb {R}^{d}\) that minimizes the sum of squared distances to its nearest \((1-\alpha )n\) points. More generally, the \(\ell _{p}\) subspace approximation problem with outliers minimizes the sum of p-th powers of distances instead of the sum of squared distances. Even the case of \(p=2\) or robust PCA is non-trivial, and previous work requires additional assumptions on the input or generative models for it. Any multiplicative approximation algorithm for the subspace approximation problem with outliers must solve the robust subspace recovery problem, a special case in which the \((1-\alpha )n\) inliers in the optimal solution are promised to lie exactly on a k-dimensional linear subspace. However, robust subspace recovery is Small Set Expansion (SSE)-hard, and known algorithmic results for robust subspace recovery require strong assumptions on the input, e.g., any d outliers must be linearly independent.