Efficient importance sampling for binary contingency tables

Abstract

Importance sampling has been reported to produce algorithms with excellent empirical performance in counting problems. However, the theoretical support for its e¢ ciency in these applications has been very limited. In this paper, we propose a methodology that can be used to design e¢ cient importance sampling algorithms for counting and test their e¢ ciency rigorously. We apply our techniques after transforming the problem into a rare-event simulation problem –thereby connecting complexity analysis of counting problems with e¢ ciency in the context of rare-event simulation. As an illustration of our approach, we consider the problem of counting the number of binary tables with …xed column and row sums, cj’s and ri’s respectively, and total marginal sums d = P j cj. Assuming that maxj cj = o d 1=2 � , P c 2 = O(d) and the rj’s are bounded we show that a suitable importance sampling algorithm (proposed by Chen, Diaconis, Holmes and Liu (2005)) requires O(d 2 2 � 1 ) operations to produce an estimate that has -relative error with probability 1 �. In addition, if maxj cj = o d 1=4 �0 � for some �0 > 0, the same coverage can be guaranteed with O d 2 2 log � 1 �� operations.