The Implications of Alternative Allocation Criteria in Adaptive Design for Panel Surveys

Abstract

Abstract Adaptive survey designs can be used to allocate sample elements to alternative data collection protocols in order to achieve a desired balance between some quality measure and survey costs. We compare four alternative methods for allocating sample elements to one of two data collection protocols. The methods differ in terms of the quality measure that they aim to optimize: response rate, R-indicator, coefficient of variation of the participation propensities, or effective sample size. Costs are also compared for a range of sample sizes. The data collection protocols considered are CAPI single-mode and web-CAPI sequential mixed-mode. We use data from a large experiment with random allocation to one of these two protocols. For each allocation method we predict outcomes in terms of several quality measures and costs. Although allocating the whole sample to single-mode CAPI produces a higher response rate than allocating the whole sample to the mixed-mode protocol, we find that two of the targeted allocations achieve a better response rate than single-mode CAPI at a lower cost. We also find that all four of the targeted designs out-perform both single-protocol designs in terms of representativity and effective sample size. For all but the smallest sample sizes, the adaptive designs bring cost savings relative to CAPI-only, though these are fairly modest in magnitude.