Reconsidering the Measurement of Proficiency in Pilot and Air Traffic Controller Radiotelephony Communication: From Construct Definition to Task Design

Abstract

The high-stakes context of international radiotelephony (RT) communication, in which pilots and air traffic controllers (ATCO) use aviation English (AE) as a lingua franca, requires a robust testing policy that is clear and fair to all stakeholders. The International Civil Aviation Organization Language Proficiency Requirements have been criticized for their lack of fit with pilots' and ATCOs' real-life communicative needs, for both native and non-native speakers of English (Douglas, 2014; Kim, 2012). This dissertation investigates and specifies the proficiency construct (awareness, knowledge, skills, and attitudes) in pilot-ATCO intercultural RT, following Fulcher and Davidson's (2007) test development framework, and drawing on theoretical and empirical studies in the domains of Aviation English, English as a Lingua Franca, Intercultural awareness, and Interactional competence. It is guided by the following overarching research question: What are the communicative demands of pilots and ATCOs involved in intercultural RT communications; how can they be specified within a construct framework and operationalized as test tasks? In order to address this question, a multiphase mixed methods (MM) research design (Creswell, 2014) with three integrated phases is applied. In phase one, the intercultural RT context is explored (MM exploratory study). In phase two, models of language use relevant to this aviation workplace are proposed; and, key construct components are specified and validated by 128 aviation stakeholders (qualitative study). In the third/final phase, draft tasks that operationalize the identified RT construct are designed and pilot tested with AE testing experts (convergent parallel MM study). Integration of all findings allows for meta-inferences regarding the research question and underscores 1) the value of MM research designs in addressing such complex questions, and 2) the role of domain experts in: exploring a target testing context, specifying and validating a construct, and designing and pilot testing tasks to generate the evidence needed to make valid inferences about test-takers' specific purpose language ability. A broader view of professional communicative competence for intercultural RT suggests an encouraging perspective in a highly specialized occupational domain, has implications for aviation safety, and serves as an example of how non-linguistic competencies could be operationalized in contexts other than aviation.