Assessing the Effect of Calibration on Nonresponse Bias in the 2005 ARMS Phase III Sample Using 2002 Census of Agriculture Data

Abstract

Agricultural Resource Management Survey (ARMS) is conducted by the USDA and collects detailed economic data from US producers. As with most surveys, ARMS suffers from unit nonresponse (70.5% in the 2005) with the potential to introduce nonresponse bias. Nonresponse bias was assessed by matching records sampled for the ARMS with those from the 2002 Census. Mean relative bias was assessed for 17 variables by comparing estimates based on census data for all ARMS cases (respondents and nonrespondents) versus ARMS respondents, using both uncalibrated and calibrated base sample weights. Nine of the 17 had significant bias using the ARMS base weights. The ARMS calibration weights reduced the bias so that it was no longer significantly different from zero in 90% of the study variables. This suggests that calibration is an effective tool in reducing nonresponse bias to acceptable levels.