Fast quantum algorithms for least squares regression and statistic leverage scores

Abstract

Least squares regression is the simplest and most widely used technique for solving overdetermined systems of linear equations Ax=b, where A∈Rn×p has full column rank and b∈Rn. Though there is a well known unique solution x⁎∈Rp to minimize the squared error ‖Ax−b‖22, the best known classical algorithm to find x⁎ takes time Ω(n), even for sparse and well-conditioned matrices A, a fairly large class of input instances commonly seen in practice. In this paper, we design an efficient quantum algorithm to generate a quantum state proportional to |x⁎〉. The algorithm takes only O(log⁡n) time for sparse and well-conditioned A. When the condition number of A is large, a canonical solution is to use regularization. We give efficient quantum algorithms for two regularized regression problems, including ridge regression and δ-truncated SVD, with similar costs and solution approximation.Given a matrix A∈Rn×p of rank r with SVD A=UΣVT where U∈Rn×r, Σ∈Rr×r and V∈Rp×r, the statistical leverage scores of A are the squared row norms of U, defined as si=‖Ui‖22, for i=1,…,n. The matrix coherence is the largest statistic leverage score. These quantities play an important role in many machine learning algorithms. The best known classical algorithm to approximate these values runs in time Ω(np). In this work, we introduce an efficient quantum algorithm to approximate si in time O(log⁡n) when A is sparse and the ratio between A's largest singular value and smallest non-zero singular value is constant. This gives an exponential speedup over the best known classical algorithms. Different than previous examples which are mainly modern algebraic or number theoretic ones, this problem is linear algebraic. It is also different than previous quantum algorithms for solving linear equations and least squares regression, whose outputs compress the p-dimensional solution to a log⁡(p)-qubit quantum state.