Evaluation of oral performance in outsourced call centres: An exploratory case study

Abstract

This case study discusses the development and use of an oral performance assessment instrument intended to evaluate Filipino agents’ customer service transactions with callers from the United States (US). The design and applications of the instrument were based on a longitudinal, qualitative observation of language training and customer service support practices of Philippine-based agents employed by a US-owned call centre company. Although language training in Philippine call centers continues to improve (Lockwood, 2012), there are still clear limitations to how the oral performance of Filipino agents is evaluated internally by call centre companies. Specialized assessment instruments, following ESP/EOP norms, broadly used by the industry are still relatively untested and many call centers maintain their own metrics that often measure agents’ language use and service quality separately (Friginal, 2007, 2009). In this study, the assessment instrument was adapted from the Melbourne Medical Students’ Diagnostic Speaking Scale (Grove & Brown, 2001) and further developed to include ESP/EOP approaches in this context of inter-cultural communication. A conveniently sampled set of recorded calls (N=100) across different task categories (e.g., troubleshooting interactions, product inquiry) was used to test the instrument for initial reliability measures. Results and analysis of the instrument’s context suitability and limitations are discussed below.