Measuring the success of scaleable open online courses

Abstract

Purpose – The purpose of this paper is to propose measures of online open course success for non-commercial institutional providers of massive open online courses (MOOCs) and other scaleable open online courses (SOOCs). Design/methodology/approach – The measures are derived from the characteristics of open online courses, existing knowledge about open online course providers and users and their motivations, and current practice in MOOC evaluation and data analytics. Findings – Current practices for evaluation of open online courses are dominated by MOOC analytics which provide insights into user demographics and behaviour with some implications for evaluation of reach and course design but leaving many unknowns. Measures for evaluation of success at the institutional level can be derived from institutional goals for open online courses. Success from the point of view of teachers and technical teams involved in design, development and delivery of open online courses can be derived from team members’ expectations, resources and satisfaction as well as measures of cost and effort compared to budget and benchmarks. Users are classified as registrants (information seekers, window shoppers, samplers), downloaders and participants (starters, partial participants and full participants who are further divided into auditing, active and certificate takers); different measures are appropriate for each group. Practical implications – Practitioners and researchers must consider a variety of levels and indicators of success to adequately evaluate open online courses. Tables in the text propose measures, methods, timing and roles. Originality/value – This is the first published paper to take a holistic view of open online course evaluation and propose detailed measures.