Membership testing for Bernoulli and tail-dependence matrices

Abstract

Testing a given matrix for membership in the family of Bernoulli matrices is a long-standing problem; the many applications of Bernoulli vectors in computer science, finance, medicine, and operations research emphasize its practical relevance. After the three-variate case was covered by Chaganty and Joe (2006) by means of partial correlations, a novel approach towards this problem was taken by Fiebig et al. (2017) for low-dimensional settings, i.e., d≤6. The latter authors were the first to exploit the close relationship between the probabilistic world of Bernoulli matrices and the well-studied correlation and cut polytopes. Inspired by this approach, we use results from Deza and Laurent (1997), Embrechts et al. (2016), and Fiebig et al. (2017) in a pre-phase of a novel algorithm to check necessary as well as sufficient conditions, before actually testing a matrix for Bernoulli compatibility. In our main approach, however, we build upon an early attempt by Lee (1993) based on the vertex representation of the correlation polytope and directly solve the corresponding linear program. To deal appropriately with the issue of exponentially many primal variables, we propose a specifically tailored column generation method. A straightforward, yet novel, analysis of the arising subproblem of determining the most violated dual constraint in the column generation process leads to an exact algorithm for membership testing. Although the membership problem is known to be NP-complete, we observe very promising performance up to dimension d=40 on a broad variety of test problems. An important byproduct of the numerical solution is a representation for Bernoulli matrices with (only) d2 many vertices that gives rise to an efficient simulation routine.