Dispositional Learning Analytics Research Articles

Dispositional learning analytics and formative assessment: an inseparable twinship

Educational innovations, particularly those in online education and technology-enhanced learning, some accelerated by the recent pandemic, take centre stage in this journal. Examples include the resurgence of the flipped classroom methodology, supported by instructional technology, the utilization of formative assessment with technological assistance to provide effective learning feedback, and the integration of learning analytics within technology-enhanced learning. Despite empirical studies addressing these advancements individually, they tend to focus predominantly on the initial phase of learning feedback: where the learner currently stands. This paper contends that isolated depictions of formative assessment overlook the broader progress in technology-enhanced learning. Our contribution highlights the significance of dispositional learning analytics, which combines learning and learner data to offer insights into learners' current status, future trajectory, and methods to facilitate their advancement. We advocate for strengthening the integration of formative assessment data within learning analytics to fully harness their combined potential. An empirical study is presented to illustrate the predictive value of formative practice behaviour on module performance and explore the link between students' learning dispositions and their engagement with e-tutorials. By delineating student profiles based on practice behaviour and examining variations in learning dispositions, our aim is to enhance our comprehension of effectively supporting learners in technology-enhanced educational environments.

Understanding self-regulation strategies in problem-based learning through dispositional learning analytics

In the ongoing discussion about how learning analytics can effectively support self-regulated student learning and which types of data are most suitable for this purpose, this empirical study aligns with the framework who advocated the inclusion of both behavioral trace data and survey data in learning analytics studies. By incorporating learning dispositions in our learning analytics modeling, this research aims to investigate and understand how students engage with learning tasks, tools, and materials in their academic endeavors. This is achieved by analyzing trace data, which captures digital footprints of students’ interactions with digital tools, along with survey responses from the Study of Learning Questionnaire (SLQ), to comprehensively examine their preferred learning strategies. Additionally, the study explores the relationship between these strategies and students’ learning dispositions measured at the start of the course. An innovative aspect of this investigation lies in its emphasis on understanding how learning dispositions act as antecedents and potentially predict the utilization of specific learning strategies. The data is scrutinized to identify patterns and clusters of such patterns between students’ learning disposition and their preferred strategies. Data is gathered from two cohorts of students, comprising 2,400 first year students. This analytical approach aims to uncover predictive insights, offering potential indicators to predict and understand students’ learning strategy preferences, which holds value for teachers, educational scientists, and educational designers. Understanding students’ regulation of their own learning process holds promise to recognize students with less beneficial learning strategies and target interventions aimed to improve these. A crucial takeaway from our research underscores the significance of flexibility, which entails the ability to adjust preferred learning strategies according to the learning environment. While it is imperative to instruct our students in deep learning strategies and encourage autonomous regulation of learning, this should not come at the expense of acknowledging situations where surface strategies and controlled regulation may prove to be more effective.

Modelling Temporality in Person- and Variable-Centred Approaches

Learning analytics needs to pay more attention to the temporal aspect of learning processes, especially in self-regulated learning (SRL) research. In doing so, learning analytics models should incorporate both the duration and frequency of learning activities, the passage of time, and the temporal order of learning activities. However, where this exhortation is widely supported, there is less agreement on its consequences. Does paying tribute to temporal aspects of learning processes necessarily imply that event-based models are to replace variable-based models, and analytic discovery methods substitute traditional statistical methods? We do not necessarily require such a paradigm shift to give temporal aspects their position. First, temporal aspects can be integrated into variable-based models that apply statistical methods by carefully choosing appropriate time windows and granularity levels. Second, in addressing temporality in learning analytic models that describe authentic learning settings, heterogeneity is of crucial importance in both variable- and event-based models. Variable-based person-centred modelling, where a heterogeneous sample is split into homogeneous subsamples, is suggested as a solution. Our conjecture is illustrated by an application of dispositional learning analytics, describing authentic learning processes over an eight-week full module of 2,360 students.

Academic boredom(s): a person-centered investigation.

Should we refer to boredom or boredoms? Research on the emotion of boredom sets itself apart from studies on other emotions by posing the question: is boredom a singular concept or does it have multiple facets? In this manuscript presenting empirical research on academic boredom, our aim is to demonstrate the justification for claiming a distinct position. Person-centered models examining university students' achievement emotions reveal the existence of multiple types of boredom, in contrast to other learning emotions that are typically represented as singular constructs. Using data generated by dispositional learning analytics applications, we further investigate the progression of learning boredom over time, exploring the impact of the pandemic and analyzing how various student learning aptitudes, such as mindsets, epistemological beliefs, epistemic emotions, learning motivation, engagement, as well as demographic factors like gender and culture, can be considered as potential antecedents or triggers of boredom. Consistent with the control-value theory of achievement emotions, we conclude that control and value constructs serve as proximal antecedents of boredom, alongside epistemic boredom as a distal antecedent. However, the relationships between boredom and its antecedents exhibit notable variations across different types of boredom.

Learning analytics and flipped learning in online teaching for supporting preservice teachers’ learning of quantitative research methods

Research methods, including those of a quantitative nature, are an important part of preservice teacher training in Finland. However, quantitative research methods are considered challenging, often feared, and even hated among preservice teachers. This may be due to previous negative experiences and emotions associated with their use, which also influence other aspects of learning such as self-regulation, self-efficacy, and orientations. Given such circumstances, new ways to teach and support the learning of quantitative methods are needed. Here, we investigate the self-regulation, self-efficacy, orientations, and emotions of preservice teachers (N = 38) enrolled in a quantitative methods online course incorporating learning analytics and a flipped learning approach. Dispositional learning analytics data from five measurement points were used, and data were analyzed via descriptive statistics, internal consistency (Cronbach alpha), bootstrapped paired sample t-test (between first and final measurement point), and profiles based on mean. The results demonstrate that in this teaching context, preservice teachers’ time management skills can be improved, and task avoidance, anxiety, and boredom towards quantitative methods decreased. The meaning of these results from the teaching context perspective are also examined, as are the limitations and implications of this study.

Dispositional Learning Analytics for Supporting Individualized Learning Feedback

An important goal of learning analytics (LA) is to improve learning by providing students with meaningful feedback. Feedback is often generated by prediction models of student success using data about students and their learning processes based on digital traces of learning activities. However, early in the learning process, when feedback is most fruitful, trace-data-based prediction models often have limited information about the initial ability of students, making it difficult to produce accurate prediction and personalized feedback to individual students. Furthermore, feedback generated from trace data without appropriate consideration of learners’ dispositions might hamper effective interventions. By providing an example of the role of learning dispositions in an LA application directed at predictive modeling in an introductory mathematics and statistics module, we make a plea for applying dispositional learning analytics (DLA) to make LA precise and actionable. DLA combines learning data with learners’ disposition data measured through for example self-report surveys. The advantage of DLA is twofold: first, to improve the accuracy of early predictions; and second, to link LA predictions with meaningful learning interventions that focus on addressing less developed learning dispositions. Dispositions in our DLA example include students’ mindsets, operationalized as entity and incremental theories of intelligence, and corresponding effort beliefs. These dispositions were inputs for a cluster analysis generating different learning profiles. These profiles were compared for other dispositions and module performance. The finding of profile differences suggests that the inclusion of disposition data and mindset data, in particular, adds predictive power to LA applications.

Subjective data, objective data and the role of bias in predictive modelling: Lessons from a dispositional learning analytics application.

For decades, self-report measures based on questionnaires have been widely used in educational research to study implicit and complex constructs such as motivation, emotion, cognitive and metacognitive learning strategies. However, the existence of potential biases in such self-report instruments might cast doubts on the validity of the measured constructs. The emergence of trace data from digital learning environments has sparked a controversial debate on how we measure learning. On the one hand, trace data might be perceived as "objective" measures that are independent of any biases. On the other hand, there is mixed evidence of how trace data are compatible with existing learning constructs, which have traditionally been measured with self-reports. This study investigates the strengths and weaknesses of different types of data when designing predictive models of academic performance based on computer-generated trace data and survey data. We investigate two types of bias in self-report surveys: response styles (i.e., a tendency to use the rating scale in a certain systematic way that is unrelated to the content of the items) and overconfidence (i.e., the differences in predicted performance based on surveys' responses and a prior knowledge test). We found that the response style bias accounts for a modest to a substantial amount of variation in the outcomes of the several self-report instruments, as well as in the course performance data. It is only the trace data, notably that of process type, that stand out in being independent of these response style patterns. The effect of overconfidence bias is limited. Given that empirical models in education typically aim to explain the outcomes of learning processes or the relationships between antecedents of these learning outcomes, our analyses suggest that the bias present in surveys adds predictive power in the explanation of performance data and other questionnaire data.

Individual differences in the preference for worked examples: Lessons from an application of dispositional learning analytics

SummaryWorked‐examples have been established as an effective instructional format in problem‐solving practices. However, less is known about variations in the use of worked examples across individuals at different stages in their learning process in student‐centred learning contexts. This study investigates different profiles of students' learning behaviours based on clustering learning dispositions, prior knowledge, and the choice of feedback strategies in a naturalistic setting. The study was conducted on 1,072 students over an 8‐week long introductory mathematics course in a blended instructional format. While practising exercises in a digital learning environment, students can opt for tutored problem solving, untutored problem solving, or call worked examples. The results indicated six distinct profiles of learners regarding their feedback preferences in different learning phases. Finally, we investigated antecedents and consequences of these profiles and investigated the adequacy of used feedback strategies concerning ‘help‐abuse’. This research indicates that the use of instructional scaffolds as worked‐examples or hints and the efficiency of that use differs from student to student, making the attempt to find patterns at an overall level a hazardous endeavour.

Supporting the less-adaptive student: the role of learning analytics, formative assessment and blended learning

How can we best facilitate students most in need of learning support, entering a challenging quantitative methods module at the start of their bachelor programme? In this empirical study into blended learning and the role of assessment for and as learning, we investigate learning processes of students with different learning profiles. Specifically, we contrast learning episodes of two cluster analysis-based profiles, one profile more directed to deep learning and self-regulation, the other profile more directed toward stepwise learning and external regulation. In a programme based on problem-based learning, where students are supposedly being primarily self-directed, this first profile is regarded as being of an adaptive type, with the second profile less adaptive. Making use of a broad spectrum of learning and learner data, collected in the framework of a dispositional learning analytics application, we compare these profiles on learning dispositions, such as learning emotions, motivation and engagement, learning performance and trace variables collected from the digital learning environments. Outcomes suggest that the blended design of the module with the digital environments offering many opportunities for assessment of learning, for learning and as learning together with actionable learning feedback, is used more intensively by students of the less adaptive profile.

Student profiling in a dispositional learning analytics application using formative assessment

How learning disposition data can help us translating learning feedback from a learning analytics application into actionable learning interventions, is the main focus of this empirical study. It extends previous work (Tempelaar, Rienties, & Giesbers, 2015), where the focus was on deriving timely prediction models in a data rich context, encompassing trace data from learning management systems, formative assessment data, e-tutorial trace data as well as learning dispositions. In this same educational context, the current study investigates how the application of cluster analysis based on e-tutorial trace data allows student profiling into different at-risk groups, and how these at-risk groups can be characterized with the help of learning disposition data. It is our conjecture that establishing a chain of antecedent-consequence relationships starting from learning disposition, through student activity in e-tutorials and formative assessment performance, to course performance, adds a crucial dimension to current learning analytics studies: that of profiling students with descriptors that easily lend themselves to the design of educational interventions.

Adding dispositions to create pedagogy-based Learning Analytics

This empirical study aims to demonstrate how Dispositional Learning Analytics (DLA) can provide a strong connection between Learning Analytics (LA) and pedagogy. Where LA based models typically do well in predicting course performance or student drop-out, they lack actionable data in order to easily connect model predictions with educational interventions. Using a showcase based on learning processes of 1080 students in a blended introductory quantitative course, we analysed the use of worked-out examples by students. Our method is to combine demographic and trace data from learning-management systems with self-reports of several contemporary social-cognitive theories. Students differ not only in the intensity of using worked-out examples but also in how they positioned that usage in their learning cycle. These differences could be described both in terms of differences measured by LA trace variables and by differences in students’ learning dispositions. We conjecture that using learning dispositions with trace data has significant advantages for understanding student’s learning behaviours. Rather than focusing on low user engagement, lessons learned from LA applications should focus on potential causes of suboptimal learning, such as applying ineffective learning strategies.

Towards Actionable Learning Analytics Using Dispositions

Studies in the field of learning analytics (LA) have shown students’ demographics and learning management system (LMS) data to be effective identifiers of “at risk” performance. However, insights generated by these predictive models may not be suitable for pedagogically informed interventions due to the inability to explain why students display these behavioral patterns. Therefore, this study aims at providing explanations of students’ behaviors on LMS by incorporating dispositional dimensions (e.g., self-regulation and emotions) into conventional learning analytics models. Using a combination of demographic, trace, and self-reported data of eight contemporary social-cognitive theories of education from 1,069 students in a blended introductory quantitative course, we demonstrate the potential of dispositional characteristics of students, such as procrastination and boredom. Our results highlight the need to move beyond simple engagement metrics, whereby dispositional learning analytics provide an actionable bridge between learning analytics and educational intervention.

In search for the most informative data for feedback generation: Learning analytics in a data-rich context

Learning analytics seek to enhance the learning processes through systematic measurements of learning related data and to provide informative feedback to learners and teachers. Track data from learning management systems (LMS) constitute a main data source for learning analytics. This empirical contribution provides an application of Buckingham Shum and Deakin Crick’s theoretical framework of dispositional learning analytics: an infrastructure that combines learning dispositions data with data extracted from computer-assisted, formative assessments and LMSs. In a large introductory quantitative methods module, 922 students were enrolled in a module based on the principles of blended learning, combining face-to-face problem-based learning sessions with e-tutorials. We investigated the predictive power of learning dispositions, outcomes of continuous formative assessments and other system generated data in modelling student performance of and their potential to generate informative feedback. Using a dynamic, longitudinal perspective, computer-assisted formative assessments seem to be the best predictor for detecting underperforming students and academic performance, while basic LMS data did not substantially predict learning. If timely feedback is crucial, both use-intensity related track data from e-tutorial systems, and learning dispositions, are valuable sources for feedback generation.