Learning Juntas in the Presence of Noise

Abstract

AbstractThe combination of two major challenges in algorithmic learning is investigated: dealing with huge amounts of irrelevant information and learning from noisy data. It is shown that large classes of Boolean concepts that only depend on a small fraction of their variables—so-called juntas—can be learned efficiently from uniformly distributed examples that are corrupted by random attribute and classification noise. We present solutions to cope with the manifold problems that inhibit a straightforward generalization of the noise-free case. Additionally, we extend our methods to non-uniformly distributed examples and derive new results for monotone juntas in this setting. We assume that the attribute noise is generated by a product distribution. Otherwise fault-tolerant learning is in general impossible which follows from the construction of a noise distribution P and a concept class \(\mathcal{C}\) such that it is impossible to learn \(\mathcal{C}\) under P-noise.KeywordsBoolean FunctionRelevant VariableConcept ClassTruth TableIrrelevant InformationThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.