Estimating heterogeneous reactions to experimental treatments

Abstract

Frequently in experiments there is not only variance in the reaction of participants to treatment. The heterogeneity is patterned: discernible types of participants react differently. In principle, a finite mixture model is well suited to simultaneously estimate the probability that a given participant belongs to a certain type, and the reaction of this type to treatment. Yet finite mixture models may need more data than the experiment provides. The approach in principle requires ex ante knowledge about the number of types. Finite mixture models make distributional assumptions that one may not feel comfortable with. They are hard to estimate for panel data, which is what experiments often generate. For experiments with repeated measurements, this paper offers a simple two-step alternative that is much less data hungry, that allows to find the number of types in the data, that does not make distributional assumptions about the type space, and that allows for the estimation of panel data models. It combines machine learning methods with classic frequentist statistics.