Reliability and Validity in Nonmarket Valuation

Abstract

A central goal of nonmarket valuation studies is to provide accurate value estimates. We suggest a systematic, comprehensive approach to accuracy assessment using the twin concepts of reliability and validity. Reliability has to do with variance; validity has to do with bias. If procedures applied in a valuation study are produce erratic results, accuracy suffers even if those procedures are unbiased. And, even if the procedures in question are reliable, they will be less useful if the procedures applied produce large biases in the estimates. We adapt the general concepts of reliability and validity to apply to nonmarket valuation studies. As in many other disciplines, the concept to be measured, the “true values,” is unobservable. Hence, criteria must be developed to serve as indicators of accuracy. Reliability is typically observed using the estimated standard error of the mean from repeated estimates of value. Validity is assessed using the concepts of content validity, construct validity, and criterion validity; what we refer to as the “three Cs” of validity. After fleshing out these concept, we illustrate how they can be applied using two case studies. Contingent valuation serves as an example of stated preference methods. The travel cost method provides a case study of revealed preference methods.