Efficient Construction of Rigid Matrices Using an NP Oracle

Abstract

For a matrix $H$ over a field $\mathbb{F}$, its rank-$r$ rigidity, denoted by $\mathscr{R}_{H}(r)$, is the minimum Hamming distance from $H$ to a matrix of rank at most $r$ over $\mathbb{F}$. A central open challenge in complexity theory is to give explicit constructions of rigid matrices for a variety of parameter settings. In this work, building on Williams' seminal connection between circuit-analysis algorithms and lower bounds [J. ACM, 61 (2014), 2], we give a construction of rigid matrices in $P}^{NP}}$. Letting $q = p^r$ be a prime power, we show that there is an absolute constant $\delta>0$ such that, for all constants $\varepsilon >0$, there is a $P}^{NP}}$ machine $M$ such that, for infinitely many $N$'s, $M(1^N)$ outputs a matrix $H_N \in \{0,1\}^{N \times N}$ with $\mathscr{R}_{H_N}(2^{(\log N)^{1/4 - \varepsilon}}) \ge \delta \cdot N^2$ over $\mathbb{F}_q$. Using known connections between matrix rigidity and other topics in complexity theory, we derive several consequences of our constructions, including that there is a function $f \in TIME}[2^{(\log n)^{\omega(1)}}]^{NP}}$ such that $f \notin PH}^{cc}}$. Previously, it was open whether $E}^{NP}} \subset PH}^{cc}}$. For all $\varepsilon >0$, there is a $P}^{NP}}$ machine $M$ such that, for infinitely many $N$'s, $M(1^N)$ outputs an $N \times N$ matrix $H_N \in \{0,1\}^{N \times N}$ whose linear transformation requires depth-2 $\mathbb{F}_q$-linear circuits of size $\Omega(N \cdot 2^{(\log N)^{1/4 - \varepsilon}})$. The previous best lower bound for an explicit family of $N \times N$ matrices over $\mathbb{F}_q$ was only $\Omega(N \log^2 N / (\log\log N)^2)$, for asymptotically good error correcting codes.