Perceived Session Quality Scale: What contributes to the quality of synchronous online education?

Abstract

AbstractThe purpose of this research was to construct an instrument to measure participants’ satisfaction with synchronous online education, explore its psychometric properties and explore differences among sessions based on their format and content. The quantitative analyses employed factor analysis in conjunction with item response theory for validation purposes and a (multivariate) analysis of variance with multilevel modelling for comparison purposes (N = 433). The qualitative analysis relied on classical content analysis of 303 open‐question feedback responses classified as Promoters or Detractors. Eight out of 10 questions from the initial item pool were retained for the final scale. In contrast with current knowledge about synchronous online education, interactivity was related to overall perceived session quality the least compared with other aspects included. Qualitative research provided pragmatic insights about the participants’ perspective on session quality and a comprehensive map of potentially relevant factors that could be a meaningful focus of future iterations of research. A relatively small and conceptually homogeneous pool of items prevented the extraction of additional factors due to discriminant validity issues. In future research, a larger and more comprehensive pool of items should be used as a starting point for constructing a scale, and if possible, longitudinal measures of learning transfer included as well. Educators can immediately make use of the practical suggestions in their instructional design, use the Perceived Session Quality Scale as a brief screening instrument to evaluate their sessions, and benchmark their quality in light of percentile scores provided for various types of sessions. The major contributions of this paper are the construction of a short, generalizable, and psychometrically valid tool for (synchronous online) education screening assessment—the Perceived Session Quality Scale—and an empirical mapping of potentially relevant aspects that contribute to perceived session quality.