An Analysis of Statistical Errors in Contingent Valuation Surveys

Abstract

Environmental economists using contingent valuation (CV) surveys are often interested in estimating the population willingness to pay (WTP) for some intervention from a sample of individuals randomly drawn from some population of interest. These surveys collect information on bids or WTP amounts and various demographic and socioeconomic characteristics about the respondents that is used to estimate descriptive statistics (such as mean or median WTP) or regression equations (such as bid or WTP functions). Statistical errors can occur when either the framework from which the sample was drawn is ignored or when the sampled elements are used to incorrectly infer relationships for a target. These errors will often cause biased WTP estimates, although the magnitude and direction of the bias are often difficult to identify. In this paper, standard methods for testing and addressing these potential sources of error are presented and alternative, and often less restrictive, methods are presented and applied to a contingent valuation study of environmentally friendly changes in the Glen Canyon Dam's operations (Welsh, et al, 1995). Great strides have been made to reduce the potential biases in CV surveys. For the better part of the last 20 years, research in environmental economics has focused on reducing potential sources of biases such as embedding, information and scope effects. But research on who and what the respondents and nonrespondents represent has often taken a back seat and in practice is either assumed away or simply overlooked (e.g., nonrespondents are assumed to either (i) behave like respondents or (ii) behave in a particular way). Two types of statistical errors associated with this final stage include sampling error and response error (a combination of nonresponse and selection errors). Sampling errors can arise when the sample drawn does not directly reflect the population of interest, most commonly occurring when incorrect sampling and survey methods are used, most notably ignoring sample weights when complex samples are drawn. Sample weights effectively represent the number of individuals in the population that each sampled individual represents. Moreover, sample weights can be used to test for non-ignorable sampling designs that might cause selection errors and can be used to protect against model misspecification. The role of sample weights in the statistical analysis of survey data, however, is subject to considerable debate among theoretical statisticians despite its widespread use by applied statisticians. Response errors occur when sampled elements are used to infer relationships for a population where the sampled elements do not represent the target population. Regardless of the sampling structure, systematic differences between respondents and nonrespondents will usually invalidate population inferences based solely on survey data from respondents. Research in marketing and psychology has determined that nonrespondents will often differ from respondents on demographic characteristics, socioeconomic status, and attitudes and beliefs, especially those related to the survey in question. Additionally, selection error occurs as a result of respondents (and nonrespondents) censoring their bids. Selection errors are likely to occur for two reasons: (i) removing protest and other missing bids as if they were outliers, and (ii) nonrespondents censoring themselves. To adjust for these problems, most statistical approaches are forced to assume some sort of model relating the likelihood of response to willingness to pay. After applying the alternative methods to the data from the Glen Canyon Dam Study, this paper will show that the results from these methods differ enough to indicate that whether or not an intervention is deemed worthwhile could depend on the error correction method used. Since the more advanced and less restrictive methods generally provide more accurate results, environmental economists should be well versed in these methods or work closely with survey statisticians to obtain the most accurate estimates of population WTP.