Classification into Kullback–Leibler balls in exponential families

Abstract

A classification procedure for a two-class problem is introduced and analyzed, where the classes of probability density functions within a regular exponential family are represented by left-sided Kullback–Leibler balls of natural parameter vectors. If the class membership is known for a finite number of densities, only, classes are defined by constructing minimal enclosing left-sided Kullback–Leibler balls, which are seen to uniquely exist. A connection to Chernoff information between distributions is pointed out.