Correcting Sample Selection Bias of Historical Digital Trace Data: Inverse Probability Weighting (IPW) and Type II Tobit Model

Abstract

ABSTRACT Digital trace data have become one of the central pillars of media research methods. Despite the opportunities for better understanding individual users’ true behaviors in the personalized media environment, many scholars have pointed out the potential for bias in trace data collections, questioning the generalizability of findings based on them. In this study, we propose two statistical bias correction methods–Inverse Probability Weighting (IPW) and Type II Tobit, which are designed to remedy selection bias of inference from digital trace data donated by research participants. Applying these methods to Facebook take-out data, we demonstrate how the correction methods can change estimated effect sizes, which is important for the translation of academic findings into real-world impacts. We conduct two simulation studies, one under fully synthetic and another under partially simulated conditions, and find that Type II Tobit generally provides a more robust and cost-efficient correction method for digital trace data.