Distinguishing Normative Processes From Noise

Abstract

This research evaluates the relative merits of two established and two newly proposed methods for modeling impressions of social events: stepwise regression, ANOVA, Bayesian model averaging, and Bayesian model sampling. Models generated with each method are compared against a ground truth model to assess performance at variable selection and coefficient estimation. We also assess the theoretical impacts of different modeling choices. Results show that the ANOVA procedure has a significantly lower false discovery rate than stepwise regression, whereas Bayesian methods exhibit higher true positive rates and comparable false discovery rates to ANOVA. Bayesian methods also generate coefficient estimates with less bias and variance than either stepwise regression or ANOVA. We recommend the use of Bayesian methods for model specification in affect control theory.