Computing Response Metrics for Online Panels

Abstract

Abstract As more researchers use online panels for studies, the need for standardized rates to evaluate these studies becomes paramount. There are currently many different ways and conflicting terminology used to compute various metrics for online panels. This paper discusses the sparse literature on how to compute response, refusal, and other rates and proposes a set of formulas and a standardized terminology that can be used to calculate and interpret these metrics for online panel studies. A description of and distinction between probability-based and volunteer opt-in panels is made since not all metrics apply to both types. A review of the existing discussion and recommendations, mostly from international organizations, is presented for background and context. In order to propose response and other metrics, the different stages involved in building an online panel are delineated. Metrics associated with these stages contribute to cumulative response rate formulas that can be used to evaluate studies using online probability-based panels. (Only completion rates can be calculated with opt-in panels.) We conclude with a discussion of the meaning of the different metrics proposed and what we think should be reported for which type of panel.