How Many Missing Answers Can Be Tolerated by Query Learners?

Abstract

We consider the model of exact learning using an equivalence oracle and an incomplete membership oracle. In this model a random subset of the learner’s membership queries is left unanswered. Our results are as follows. First, we analyze the obvious method for coping with missing answers: search exhaustively through all possible “answer patterns” associated with the unanswered queries. Thereafter, we present two specific concept classes that are efficiently learnable using an equivalence oracle and a (completely reliable) membership oracle, but are provably not polynomially learnable if the membership oracle becomes slightly incomplete. The first class demonstrates that the aforementioned method of exhaustively searching through all possible answer patterns cannot be substantially improved in general (despite its apparent simplicity). The second class demonstrates that the incomplete membership oracle can be rendered useless even if it leaves only a fraction 1/poly(n) of all queries unanswered. Finally, we present a learning algorithm for monotone DNF formulas that can cope with a relatively large fraction of missing answers (more than 60%), but is as efficient (in terms of run-time and number of queries) as the classical algorithm whose questions are always answered reliably.