Comparison of Population-Representative and Volunteer Experiments: Lessons From the Social Security Administration’s Benefit Offset National Demonstration (BOND)

Abstract

The Benefit Offset National Demonstration (BOND) tested a $1 reduction in benefits per $2 earnings increase above the level at which Social Security Disability Insurance benefits drop from full to zero under current law. BOND included a rare and large “population-representative” experiment: It applied the rule to a nationwide, random sample of beneficiaries. BOND also included a parallel “volunteer” experiment: The same offset rules applied to a nationwide random sample of volunteers. BOND’s dual experiments offer a unique opportunity to consider the merits and limitations of these two types of experiments. The findings provide a way to illustrate the limitations of volunteer experiments relative to population experiments when, as in BOND, the goal is to produce impact estimates that are valid for the entire population, not just would be volunteers. We also highlight and compare ethical issues, outreach challenges, and cost considerations for the two types of designs.