A Validity Assessment of Aggregation Methods for Multiple Key Informant Survey Data

Abstract

Comparative community researchers are often faced with the task of collecting detailed information about actions and events across a large number of communities. Key informants can play a crucial role in providing such data. For a variety of reasons, however, it is often difficult for researchers to determine the validity of key informant data. In this paper we undertake such an assessment. We restrict our analysis to those instances in which key informant data are aggregated to create a community score for specific survey questions. We assess the validity of four popular aggregation methods: the "modal yes" method, the "modal no" method, the "any yes" method, and the "partial yes" method. Our analysis suggests that the partial yes and any yes methods yield more correct classifications than the modal no and modal yes aggregation methods. We conclude with a brief discussion of the implications of our findings for community development research.