Common Information and Unique Disjointness

Abstract

We provide an information-theoretic framework for establishing strong lower bounds on the nonnegative rank of matrices by means of common information, a notion previously introduced in Wyner (IEEE Trans Inf Theory 21(2):163---179, 1975). The framework is a generalization of the one in Braverman and Moitra (Proceedings of the forty-fifth annual ACM symposium on theory of computing, pp 161---170, 2013) for the shifted uniqe disjointness (UDISJ) matrix to arbitrary nonnegative matrices. Common information is a natural lower bound for the nonnegative rank of a matrix and by combining it with Hellinger distance estimations we compute the (almost) exact common information of UDISJ partial matrix. The bounds are obtained very naturally. We also establish robustness of this estimation under random or adversarial removal of rows and columns of the UDISJ partial matrix. This robustness translates, via a variant of Yannakakis's factorization theorem, to lower bounds on the average case and adversarial approximate extension complexity of removals. We present the first family of polytopes, the hard pair introduced in Braun et al. (Math Oper Res 40(3):756---772, 2015) related to the CLIQUE problem, with high average case and adversarial approximate extension complexity of removals. The framework relies on a strengthened version of the link between information theory and Hellinger distance from Bar-Yossef (J Comput Syst Sci 68(4):702---732, 2004). We also provide an information theoretic variant of the fooling set method that allows us to extend fooling set lower bounds from extension complexity to approximate extension complexity.