Item Response Rates for the Agricultural Resource Management Survey III in 2006 and 2007

Abstract

Missing data exist. Mechanisms that determine nonresponse have been eluding survey methodologists for many years. Although significant improvements in survey instruments and data collection have been made, surveys are returned without full responses. NASS’s Agricultural Resource Management Survey III (ARMS III) is not immune to the missing data problem. Without adaptation to the processing and analysis of the dataset from traditional methods, the missing values leave room for bias in estimates. The amount of missing data can exacerbate the bias of estimates when analysis is carried out on only the responding observations. Recognizing this issue, the Office of Management and Budget released a threshold of seventy percent response rate by item in the Standards and Guidelines for Statistical Surveys. Items that do not perform at this level should be investigated to determine if the items in violation are missing at random. Data that are missing at random can be analyzed without external information about the mechanism that causes those values to be missing. For the general questionnaire versions of ARMS III in 2006 and 2007, the performance based on the item response rate and change rate for each item was assessed. Overall, the ARMS III survey performs well; few items had lower item response rates than the threshold set by OMB guidelines. However, most of the problematic items appear consistently over the two years, warranting further investigation into survey design and implementation as well the structure of the missing data.