The Long-Term Impact of Different Offline Population Inclusion Strategies in Probability-Based Online Panels: Evidence From the German Internet Panel and the GESIS Panel

Carina Cornesse,Ines Schaurer

doi:10.1177/0894439320984131

Abstract

While online panels offer numerous advantages, they are often criticized for excluding the offline population. Therefore, some probability-based online panels have developed offline population inclusion strategies. Two dominant approaches prevail: providing internet equipment and offering an alternative survey participation mode. We investigate the impact of these approaches on two probability-based online panels in Germany: the German Internet Panel, which provides members of the offline population with internet equipment, and the GESIS Panel, which offers members of the offline population to participate via postal mail surveys. In addition, we explore the impact of offering an alternative mode only to non-internet users versus also offering the alternative mode to internet users who are unwilling to provide survey data online. Albeit lower recruitment and/or panel wave participation probabilities among offliners than onliners, we find that including the offline population has a positive long-term impact on sample accuracy in both panels. In the GESIS Panel, the positive impact is particularly strong when offering the alternative participation mode to non-internet users and internet users who are unwilling to provide survey data online.

Highlights

While online panels offer numerous advantages, they are often criticized for excluding the offline population
We present our results regarding the impact of offline population inclusion in the German Internet Panel (GIP), which provides its offliners with the necessary equipment to participate online, and the GESIS Panel, which provides its offliners with a mail-mode alternative, on survey participation and sample accuracy across panel survey waves
We examined the impact of two approaches to offline population inclusion in probability-based online panels: providing members of the offline population with the necessary equipment to participate in surveys online and offering postal mail surveys as an alternative survey participation mode

Summary

Introduction

While online panels offer numerous advantages, they are often criticized for excluding the offline population. In the GESIS Panel, the positive impact is strong when offering the alternative participation mode to non-internet users and internet users who are unwilling to provide survey data online. This strategy is prevalent in nonprobability online panels, which rely on convenience samples of internet users (Callegaro et al, 2014) Such nonprobability online panels have repeatedly been shown to lead to invalid inferences from the survey data to the general population, especially with regard to univariate statistics (see Cornesse et al, 2020, for an overview; Baker et al, 2013; Coppock et al, 2018; Litman & Robinson, 2020; Mullinix et al, 2015, for opposing findings regarding experimental research and “fit-for-purpose” approaches). Some probability-based online panels implement an offline population inclusion strategy to ensure that every sample unit willing to participate in the panel has the chance to do so

Objectives

Results

Conclusion