Information Geometric Probability Models in Statistical Signal Processing

Abstract

The exponentially embedded family (EEF) of probability density functions (PDFs) is an important modeling paradigm in a number of signal processing applications including model selection and hypothesis testing. In recent years, the availability of inexpensive sensors that can gather information over wide, distributed areas, has led to many interesting challenges in distributed estimation and data fusion. In these applications there is a need for combining information extracted from data which is only available locally at individual sensors. The EEF is valuable in this context due to its unique properties with respect to the Kullback-Leibler divergence measure. We study the geometry of the EEF in the context of distributed applications where we must combine information at the PDF level. Central to this is the work of Chernoff, in particular, the notion and / or existence of an equidistant PDF between component PDFs or hypotheses. This equidistance is definedwith respect to the Kullback Leibler divergence, and has been utilized in recent years as a criterion for combining information (PDFs) in data fusion applications. Much of that work, however, has been limited to combining PDFs in a pairwise fashion. To expand on this we utilize results from Information Geometry in order to establish a formal extension of the "Chernoff point" (equidistant PDF) for the EEF of more than two components. Using well known theory of exponential families, we establish the proper existence criteria of such an extension, and show that much of the geometry involved can be well understood by drawing analogies between the EEF of three component PDFs, and the Euclidean geometry of triangles. We have shown that finding the extended Chernoff point can be guided by considering the orientation of the component PDFs, and that the use of this paradigm can lead to better ways to combine estimates in classical problems such as combining estimates of common means from separate Normal populations.