Abstract
A robust method is proposed for estimating discrete probability functions for small samples. The proposed approach introduces and minimizes a parameterized objective function that is analogous to free energy functions in statistical physics. A key feature of the method is a model of the parameter that controls the trade-off between likelihood and robustness in response to the degree of fluctuation. The method thus does not require the value of the parameter to be manually selected. It is proved that the estimator approaches the maximum likelihood estimator at the asymptotic limit. The effectiveness of the method in terms of robustness is demonstrated by experimental studies on point estimation for probability distributions with various entropies.
Highlights
For categorical observational data analysis, it is often necessary to deal with multivariate systems since variables of such data generally depend on each other
An objective function with parameter β is introduced for robust estimation, and probability functions obtained by the proposed method are shown to be formally equivalent to the canonical distributions that appear in statistical physics
The canonical distribution expressed as Equation (8) can provide some characteristic properties of ATML, which are similar to those in statistical physics
Summary
For categorical observational data analysis, it is often necessary to deal with multivariate systems since variables of such data generally depend on each other. Predictive statistical inference requires parameters that achieve low-entropy, so it is preferable to use data of many variables because the following relationship in Shannon entropies H of random variables X and Y holds if X and Y depend on each other: H ( X Y ) ≤ H ( X ) [1], where H ( X ) denotes Shannon entropy of X and H ( X Y ) denotes the conditional entropy of X given Y These are respectively defined with marginal, joint, and conditional probability mass functions P as follows [1]:. An objective function with parameter β is introduced for robust estimation, and probability functions obtained by the proposed method are shown to be formally equivalent to the canonical distributions that appear in statistical physics.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
