Exploring psychophysical measurement in landscape aesthetics: Validity, reliability and signal detection via single-versus opposing-construct rating scales, with or without zeros

Abstract

Surveys eliciting qualitative judgments of environments might have an objective psychophysical basis. They can employ numeric, single-construct, integer rating scales or ratio scales between opposing negative and positive end-point constructs. Each stimulus can be measured by different ‘averaging’ methods across subjects' ratings. The impacts of these choices on data reliability and validity were investigated. Three different rating scales were presented to different sets of subjects who rated the same stimuli. Eighty photos of various forests and timber harvests were rated for (1) scenic beauty on a 1 to 10 scale, (2) ugliness or scenic beauty on a −5 to +5 scale, or (3) the same bipolar scale but without a central zero value. Ratings from each photo from one rating scale were averaged and ‘averaged’ again by the Scenic Beauty Estimate (SBE) signal detection algorithm. Distributions of the raw and averaged ratings were examined to explore validity by conformance with psychophysical theory. Ratings from the bipolar scale without a zero exhibited the most evidence of validity across a wider range, indicating that it best conformed with discrimination of photos' perceived beauty and ugliness along a shared, psychophysical gradient. Averaged ratings from the 1 to 10 scale produced the most reliable data, while its SBEs were slightly less reliable but with more stable errors. These ratings exhibited poor validity among low-beauty photos The bipolar rating scale that included a zero value was least reliable and valid, particularly across its ratio SBEs. All rating scales' average ratings or SBEs exhibited acceptable reliability.