Abstract
In Monte Carlo based importance sampling estimations, Effective Sample Size (ESS) is an important index of simulation efficiency, since ESS can measure the divergence between the target distribution and the proposal distribution effectively, and thus is widely used to decide whether resampling is needed or not. Among several well-known variants of ESS, the Shannon entropy based perplexity has been widely used. In this paper, however, we propose a new ESS function (E-MIM) by using the message importance measure (MIM) instead of Shannon entropy. We show that E-MIM satisfies all of the five conditions for ESS generalizations. We also propose an MIM based divergence and investigate its approximation to E-MIM. Moreover, we present a resampling threshold selection method for the ratio between E-MIM and the corresponding actual sample size. Finally, we investigate the performance of E-MIM and other ESS functions through numerical simulations. By a particle filter experiment, we show that E-MIM outperforms other ESS functions in terms of mean-squared error.
Highlights
Sequential Monte Carlo (SMC), called bootstrap filter or particle filter, is an important tool for Bayesian inference [1], which is widely applied in statistics [2]–[4], signal processing [5]–[7] and economics [8]–[10]
One of the key factors for the success of the SMC method is the use of resampling, without which the SMC method will suffer from serious weight degeneracy [11]–[13]
We propose an effective sample size measure based on the message importance measure, which is referred as E-MIM
Summary
Sequential Monte Carlo (SMC), called bootstrap filter or particle filter, is an important tool for Bayesian inference [1], which is widely applied in statistics [2]–[4], signal processing [5]–[7] and economics [8]–[10]. Effective Sample Size (ESS) is a widely used criteria to measure the efficiency of different Monte Carlo methods [13],. Based on aforementioned considerations on ESS, the generalized effective sample size (G-ESS) was proposed, and five necessary conditions for G-ESS functions were presented [12]. Considering the application of discrete entropy in the assessment of ESS [12], [14] and adaptive importance sampling [24], [27], it is promising to use information theory tools in SMC. We propose an effective sample size measure based on the message importance measure, which is referred as E-MIM.
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
