Multimodal Data Fusion in Learning Analytics: A Systematic Review.

Su Mu,Xiaodi Huang,Meng Cui

doi:10.3390/s20236856

Su Mu, Xiaodi Huang + Show 1 more

Open Access

https://doi.org/10.3390/s20236856

Copy DOI

Abstract

Multimodal learning analytics (MMLA), which has become increasingly popular, can help provide an accurate understanding of learning processes. However, it is still unclear how multimodal data is integrated into MMLA. By following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, this paper systematically surveys 346 articles on MMLA published during the past three years. For this purpose, we first present a conceptual model for reviewing these articles from three dimensions: data types, learning indicators, and data fusion. Based on this model, we then answer the following questions: 1. What types of data and learning indicators are used in MMLA, together with their relationships; and 2. What are the classifications of the data fusion methods in MMLA. Finally, we point out the key stages in data fusion and the future research direction in MMLA. Our main findings from this review are (a) The data in MMLA are classified into digital data, physical data, physiological data, psychometric data, and environment data; (b) The learning indicators are behavior, cognition, emotion, collaboration, and engagement; (c) The relationships between multimodal data and learning indicators are one-to-one, one-to-any, and many-to-one. The complex relationships between multimodal data and learning indicators are the key for data fusion; (d) The main data fusion methods in MMLA are many-to-one, many-to-many and multiple validations among multimodal data; and (e) Multimodal data fusion can be characterized by the multimodality of data, multi-dimension of indicators, and diversity of methods.

Highlights

Learning analytics refers to the measurement, collection, analysis, and reporting of data about learners and their learning contexts, for understanding and optimizing learning and the environment in which it occurs [1]
We classified the types of multimodal data in MMLA into digital data, physical data, physiological data, psychometric data, and environment data
The complex relationships between multimodal data and learning indicators were the key to data fusion

Summary

Introduction

Learning analytics refers to the measurement, collection, analysis, and reporting of data about learners and their learning contexts, for understanding and optimizing learning and the environment in which it occurs [1]. The data for traditional learning analytics is usually unidimensional [2]. Only log data rather than all data generated by a learning management system are commonly used for analyzing the online learning process. Log data ignore important contextual information about learners [3]. These context data are crucial for understanding students’. Unidimensional data provide only partial information about the learning process [4,5], which makes it impossible to produce accurate results of learning analytics [6]

Objectives

Methods

Findings

Conclusion