Abstract
Robust Bayesian models are appealing alternatives to standard models, providing protection from data that contains outliers or other departures from the model assumptions. Historically, robust models were mostly developed on a case-by-case basis; examples include robust linear regression, robust mixture models, and bursty topic models. In this paper we develop a general approach to robust Bayesian modeling. We show how to turn an existing Bayesian model into a robust model, and then develop a generic computational strategy for it. We use our method to study robust variants of several models, including linear regression, Poisson regression, logistic regression, and probabilistic topic models. We discuss the connections between our methods and existing approaches, especially empirical Bayes and James–Stein estimation.
Highlights
Modern Bayesian modeling enables us to develop custom methods to analyze complex data (Gelman et al, 2014; Bishop, 2006; Murphy, 2013)
We provide a general strategy that can be used with simple models, nonconjugate models, and complex models with local and global variables (e.g., latent Dirichlet allocation (LDA))
We have discussed that robust LDA is a bursty topic model (Doyle and Elkan, 2009)
Summary
Modern Bayesian modeling enables us to develop custom methods to analyze complex data (Gelman et al, 2014; Bishop, 2006; Murphy, 2013). We use two ideas to build robust Bayesian models: localization and empirical Bayes (Efron and Morris, 1973). This is a more heterogeneous and robust model because it can explain unlikely data points by deviations in their individualized parameters This perspective is not new—it describes and generalizes many classical distributions that are used for robust modeling. For simplicity, we choose to use empirical Bayes estimate of α instead This algorithm is a major contribution of this paper; it generalizes the case-bycase algorithms of many existing robust models and expands the idea of robustness to a wider class of models, including those that rely on approximate posterior inference.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
