Narrative and the Fundamental Limitations of Quantification in Crosscultural Research

Abstract

at least the last 3 decades, international and crosscultural studies of health, religious beliefs, political attitudes, and much else of a fundamentally subjective nature have proliferated. Interest in comparative research emerges from the desire to understand how different cultural, economic, political, and social systems influence individuals' world views, beliefs, and behaviors. Yet the use of subjective information and self-reports has always presented researchers with serious problems that have to do with assessing the comparability of subjective information gathered from respondents from different cultural and linguistic communities (see the Spring 2001 issue of the Journal of Mental Health and Aging1 for several articles dealing with comparative measurement). Even within the same nation or the same cultural group, differences in such personal characteristics as socioeconomic status, age, gender, and education potentially affect the manner in which respondents interpret and respond to probes concerning their attitudes, beliefs, behavior, or health. These problems are magnified when one adds differences in language and culture to the mix of factors that potentially influence self-reported information.2 The fundamental problem arises from the fact that the privileged world of subjective experience can only be inferred by an observer from external indicators that, in the case of research instruments, are constructed on the basis of assumptions concerning cognitive processes or schemas grounded in the researcher's own culture. These may or may not reflect the cognitive realities of another culture. Proving crosscultural comparability is a challenge to the researcher, but it is a clear imperative. Quantification in heath research inherently rests on the assumption that human experience is largely comparable at least across some range of social and cultural distinctions and that individuals from different cultures can be scored on some common interval-level latent metric related to the phenomena under study. An extreme position in terms of comparison across cultures would hold that all human experience can be translated into a common set of basic cognitive and linguistic constructs and that those can be quantified. Such an assumption requires unambiguous proof, and the degree of comparability of concepts and measurement must be established on the basis of some adequate theoretical and empiric practice. The challenge arises in defining what such an adequate practice might be or perhaps whether any procedure can adequately assure meaningful quantitative comparisons at all. Statistical models can identify significant but small effects that can easily reflect measurement artifacts. A quick perusal of the literature reveals that authors commonly quantify the answers to questions about religious and other highly personal beliefs and relate those to self-reported health practices and outcomes. What is actually demonstrated even in one cultural group by such associations is often unclear. When the probes are asked to individuals from very different social classes and cultures in different languages, the results are uninterpretable. In crosscultural research, what has come to be accepted as common practice, or even considered necessary, is often not based on validated methodological procedures. For example, although back-translation has become the norm for the development of crosscultural measurement instruments, its value for improving validity has never been demonstrated. Back-translation represents a convention, and professional translators are