Production Implementation of Recurrent Neural Networks in Adaptive Instructional Systems

Abstract

This paper reviews current research on deep knowledge tracing (DKT) and discusses the benefits of using DKT in adaptive instructional systems (AIS). Namely, DKT allows for accurate measurement of ability levels across a set of attributes in a content domain and this information can be leveraged to deliver personalized content to the learner. DKT uses a recurrent neural network with long short-term memory units (RNN-LSTM), which is difficult to interpret, although provides higher prediction accuracy than Bayesian knowledge tracing (BKT) or item response theory (IRT) measurement approaches. This makes DKT ideal for learner-focused or formative assessment systems, in which the measurement of attribute proficiencies and the delivery of relevant content to promote learning is valued above the understanding of the measurement process itself. The paper focuses on practical considerations for preparing and deploying an RNN-LSTM in a production system. Namely, data demands for training the network are explored through an analysis on real data from an adaptive tutoring program, and novel methods for training the network when no data are available and for measuring learning trajectories are proposed. Finally, strategies around monitoring production prediction services are discussed, as well as tips for approaching latency, stability, and security issues in production environments. These discussions are meant to provide a researcher or data scientist with enough information to effectively collaborate with technical teams on the production implementation of RNNs, with the goal of making cutting-edge advances in DKT available to real learners.