Learning k-Modal Distributions via Testing

Constantinos Daskalakis,Rocco A Servedio,Ilias Diakonikolas

doi:10.1137/1.9781611973099.108

Abstract

Previous chapter Next chapter Full AccessProceedings Proceedings of the 2012 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA)Learning k-Modal Distributions via TestingConstantinos Daskalakis, Ilias Diakonikolas, and Rocco A. ServedioConstantinos Daskalakis, Ilias Diakonikolas, and Rocco A. Servediopp.1371 - 1385Chapter DOI:https://doi.org/10.1137/1.9781611973099.108PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAboutAbstract A k-modal probability distribution over the domain {1, …, n} is one whose histogram has at most k “peaks” and “valleys.” Such distributions are natural generalizations of monotone (k = 0) and unimodal (k = 1) probability distributions, which have been intensively studied in probability theory and statistics. In this paper we consider the problem of learning an unknown k-modal distribution. The learning algorithm is given access to independent samples drawn from the k-modal distribution p, and must output a hypothesis distribution p such that with high probability the total variation distance between p and is at most ∊. We give an efficient algorithm for this problem that runs in time poly(k, log(n), 1/ε). For , the number of samples used by our algorithm is very close (within an Õ(log(1/∊)) factor) to being information-theoretically optimal. Prior to this work computationally efficient algorithms were known only for the cases k = 0, 1 [Bir87b, Bir97]. A novel feature of our approach is that our learning algorithm crucially uses a new property testing algorithm as a key subroutine. The learning algorithm uses the property tester to efficiently decompose the k-modal distribution into k (near)-monotone distributions, which are easier to learn. Previous chapter Next chapter RelatedDetails Published:2012ISBN:978-1-61197-210-8eISBN:978-1-61197-309-9 https://doi.org/10.1137/1.9781611973099Book Series Name:ProceedingsBook Code:PR141Book Pages:xiii + 1757

Highlights

This paper considers a natural unsupervised learning problem involving k-modal distributions over the discrete domain {1, . . . , n}
It should be noted that some of these works do give efficient algorithms for the cases k = 0 and k = 1; in particular we mention the results of Birge [Bir87b, Bir97], which give computationally efficient O(log(n)/ 3)-sample algorithms for learning unknown monotone or unimodal distributions over [n] respectively. (Birge [Bir87a] showed that this sample complexity is asymptotically optimal, as we discuss below; we describe the algorithm of [Bir87b] in more detail in Section 2.2, and use it as an ingredient of our approach throughout this paper.) for these relatively simple k = 0, 1 classes of distributions the main challenge is in developing sample-efficient estimators, and the algorithmic aspects are typically rather straightforward
Birge [Bir87a] gave a sample complexity lower bound for learning monotone distributions

Summary

Introduction

This paper considers a natural unsupervised learning problem involving k-modal distributions over the discrete domain {1, . . . , n}. A distribution is k-modal if the plot of its probability density function (pdf) has at most k “peaks” and “valleys” (see Section 2.1 for a precise definition) Many researchers have studied the risk of different estimators for monotone and unimodal distributions; see for example the works of [Rao, Weg, Gro, Bir87a, Bir87b, Bir97], among many others. It should be noted that some of these works do give efficient algorithms for the cases k = 0 and k = 1; in particular we mention the results of Birge [Bir87b, Bir97], which give computationally efficient O(log(n)/ 3)-sample algorithms for learning unknown monotone or unimodal distributions over [n] respectively. Much more challenging and interesting algorithmic issues arise for the general values of k which we consider here

Results

Discussion

Conclusion