Measuring Discourse by Algorithm

Abstract

Scholars increasingly use machine learning techniques such as Latent Dirichlet Allocation (LDA) to reduce the dimensionality of textual data and to study discourse in collective bodies. However, measures of discourse based on algorithmic results typically have no intuitive meaning or obvious relationship to humanly observed discourse. Such measures of discourse must be carefully validated before relied on and interpreted. We examine several common measures of discourse based on algorithmic results, and propose a number of ways to validate them in the setting of Federal Open Market Committee meetings. We also suggest that validation techniques may be used as a principled approach to model selection and parameterization.