Measuring intangible outcomes can be problematic: The challenge of assessing learning during international short-term study experiences

Abstract

Short-term international study experiences, or study tours, aim to increase students’ global awareness, educate and empower them to be productive global citizens, and contribute to their future employability. Learning outcomes from study tours often include intangible personal characteristics or soft skills, as opposed to specific disciplinary skills and knowledge, and yet, these are not easily identified. Using an iterative Delphi process, this study aimed to understand the pedagogical experience of tour leaders (experts) and assist in future development of effective assessment. Findings include four key areas of agreement among the experts: (1) both discipline-based knowledge and the acquisition of intangible personal characteristics are important learning outcomes and are considered to be almost equally important, (2) assessment almost always occurs on tour, (3) formative assessment for learning is predominantly used (rather than learning for summative assessment to test the learning), and (4) facilitated reflection is an important way to encourage learning during a study tour. Yet, the findings also reveal uncertainty about assessing outcomes that include intangible personal characteristics and soft skills. The article concludes with pedagogical implications and recommendations for future research.