Development of cross-cultural performance criterion measures: Critical issues and practical solutions

Abstract

There is a need to carefully evaluate the criteria used to measure cross-cultural performance. Research on criteria measurement development for cross-cultural performance is needed because the current literature lacks clear, well-defined criteria and means to measure it well. This article aims to bridge that gap in three ways. First, we highlight five critical issues that should be considered with regard to cross-cultural performance. These five critical issues include: criterion conceptualization, rater source, standardization, retrospective versus in-situ ratings, and rater usability. Along with a discussion of each issue, techniques are presented to address each issue. Second, to illustrate and improve upon these issues, we present a case study on the development of cross-cultural performance as a criterion concept and corresponding criterion assessment tools in a military context. Specifically, we demonstrate how we defined a cross-cultural performance taxonomy used to develop measurable performance-based criteria. We describe a large-scale criterion development effort conducted to systematically develop and validate two criterion measures of cross-cultural performance used in military training contexts that simulate real-world situations. Third, in an effort to guide and focus how the cross-cultural performance space is defined, we recommend various approaches to future cross-cultural performance criteria development.