Finding Correlations in Subquadratic Time, with Applications to Learning Parities and the Closest Pair Problem

Abstract

Given a set of n d -dimensional Boolean vectors with the promise that the vectors are chosen uniformly at random with the exception of two vectors that have Pearson correlation coefficient ρ (Hamming distance d · 1-ρ/ 2), how quickly can one find the two correlated vectors? We present an algorithm which, for any constant ϵ>0, and constant ρ>0, runs in expected time O (n 5-ω / 4-ω+ϵ +nd) < O (n 1.62 +nd), where ω < 2.4 is the exponent of matrix multiplication. This is the first subquadratic--time algorithm for this problem for which ρ does not appear in the exponent of n , and improves upon O (n 2-O(ρ) ), given by Paturi et al. [1989], the Locality Sensitive Hashing approach of Motwani [1998] and the Bucketing Codes approach of Dubiner [2008]. Applications and extensions of this basic algorithm yield significantly improved algorithms for several other problems. Approximate Closest Pair. For any sufficiently small constant ϵ>0, given n d -dimensional vectors, there exists an algorithm that returns a pair of vectors whose Euclidean (or Hamming) distance differs from that of the closest pair by a factor of at most 1+ϵ, and runs in time O (n 2-Θ(√ϵ )). The best previous algorithms (including Locality Sensitive Hashing) have runtime O (n 2-O(ϵ) ). Learning Sparse Parities with Noise. Given samples from an instance of the learning parities with noise problem where each example has length n , the true parity set has size at most k « n , and the noise rate is η, there exists an algorithm that identifies the set of k indices in time n ω+ϵ/3 k poly(1/1-2η) < n 0.8k poly(1/1-2 η). This is the first algorithm with no dependence on η in the exponent of n , aside from the trivial O (( n k )) ≈ O ( n k ) brute-force algorithm, and for large noise rates (η > 0.4), improves upon the results of Grigorescu et al. [2011] that give a runtime of n (1+(2 η) 2 + o (1))k/2 poly (1/1-2η). Learning k -Juntas with Noise. Given uniformly random length n Boolean vectors, together with a label, which is some function of just k « n of the bits, perturbed by noise rate η, return the set of relevant indices. Leveraging the reduction of Feldman et al. [2009], our result for learning k -parities implies an algorithm for this problem with runtime n ω+ϵ/3 k poly (1/1-2η) < n 0.8k poly (1/1-2 η), which is the first runtime for this problem of the form n ck with an absolute constant c < 1. Learning k -Juntas without Noise. Given uniformly random length n Boolean vectors, together with a label, which is some function of k « n of the bits, return the set of relevant indices. Using a modification of the algorithm of Mossel et al. [2004], and employing our algorithm for learning sparse parities with noise via the reduction of Feldman et al. [2009], we obtain an algorithm for this problem with runtime n ω+ ϵ/4 k poly(n) < n 0.6k poly(n) , which improves on the previous best of n ω+1/ωk ≈ n 0.7k poly(n) of Mossel et al. [2004].