Teacher Dashboard Research Articles

I don't have time! But keep me in the loop: Co‐designing requirements for a learning analytics cockpit with teachers

AbstractBackgroundTeacher dashboards can help secondary school teachers manage online learning activities and inform instructional decisions by visualising information about class learning. However, when designing teacher dashboards, it is not trivial to choose which information to display, because not all of the vast amount of information retrieved from digital learning environments is useful for teaching. Information elicited from formative assessment (FA), though, is a strong predictor for student performance and can be a useful data source for effective teacher dashboards. Especially in the secondary education context, FA and feedback on FA, have been extensively studied and shown to positively affect student learning outcomes. Moreover, secondary teachers struggle to make sense of the information displayed in dashboards and decide on pedagogical actions, such as providing feedback to students.ObjectivesTo facilitate the provision of feedback for secondary school teachers via a teacher dashboard, this study identifies requirements for designing a Learning Analytics Cockpit (LA Cockpit), that is, (1) a teacher dashboard that provides teachers with visualisations of results from formative assessment (FA) and (2) a feedback system that supports teachers in providing feedback to students.MethodsThis study was conducted in the context of STEM classes and is based on semi‐structured co‐design interviews with German secondary school teachers. In these interviews, we first explored challenges teachers encountered in monitoring students' learning and providing feedback. Second, in the ideation phase, teachers were asked to define features an LA Cockpit for FA should have. Finally, in the evaluation phase, we provided teachers with a design template for an LA Cockpit, the LAC_Template, which was built upon our previous work and feedback theory, and asked them to evaluate and improve it. Further design requirements were derived based on the evaluation of the LAC_Template and teachers' suggestions for improvement.ResultsWe derived 16 requirements for designing an LA Cockpit for FA in secondary schools. Findings from the interviews indicated that the feedback system of an LA Cockpit should address teachers' time limitations in giving students individualised feedback. It should therefore be designed to minimise the steps required to deliver feedback. To reduce workload, teachers requested an automated reminder to send feedback, but with the ability to adjust feedback to the learning context. Such a semi‐automated feedback system can help teachers support students individually but also underline the importance of actively involving teachers in the feedback loop and giving them control when using such technologies in secondary school practice. A challenge for future teacher dashboard designs could be to find a balance between technology and teacher control that utilises the strengths of both in a beneficial combination.

MineObserver 2.0: A Deep Learning & In-Game Framework for Assessing Natural Language Descriptions of Minecraft Imagery

MineObserver 2.0 is an AI framework that uses Computer Vision and Natural Language Processing for assessing the accuracy of learner-generated descriptions of Minecraft images that include some scientifically relevant content. The system automatically assesses the accuracy of participant observations, written in natural language, made during science learning activities that take place in Minecraft. We demonstrate our system working in real-time and describe a teacher dashboard to showcase observations, both of which advance our previous work. We present the results of a study showing that MineObserver 2.0 improves over its predecessor both in perceived accuracy of the system's generated descriptions as well as in usefulness of the system's feedback. In future work, we intend improve system generated descriptions to give more teacher control and shift the system to perform continuous learning to more rapidly respond to novel observations made by learners.

Multimodal Teacher Dashboards: Challenges and Opportunities of Enhancing Teacher Insights Through a Case Study

Multimodal Teacher Dashboards: Challenges and Opportunities of Enhancing Teacher Insights Through a Case Study

Easy JavaScript Simulation (EJSS) Data Analytics for Singapore

We have integrated Easy JavaScript Simulation (EJSS) Data Analytics into the national Learning Management System for Singapore schools, known as the Singapore Student Learning Space (SLS). EJSS Data Analytics enhances the teaching and learning experience for educators and students by enabling educators to monitor and evaluate students’ interactions with interactive computer simulations. The data analytics and visualisation capabilities are delivered using the Moodle platform and version 1.3 of the specifications for Learning Tools Interoperability (LTI). In this paper, we showcase the potential for EJSS Data Analytics to identify students’ learning difficulties and misconceptions. Four examples of EJSS Data Analytics applications are provided to illustrate insights on aspects that include understanding a student’s sequential actions leading to specific task outcomes, the frequency of task attempts by each student, and the ratio of students achieving correct versus incorrect task completions. We identify five key considerations for designing the EJSS teacher dashboard. These considerations relate to Student Thought Process, Student Behaviour, Student Engagement, Student Choice, and Teacher Feedback. These five facets provide a framework for aligning our design efforts with the needs of students and teachers, also drawing upon research in data analytics for education.

Automated Gaze-Based Identification of Students’ Strategies in Histogram Tasks through an Interpretable Mathematical Model and a Machine Learning Algorithm

As a first step toward automatic feedback based on students’ strategies for solving histogram tasks we investigated how strategy recognition can be automated based on students’ gazes. A previous study showed how students’ task-specific strategies can be inferred from their gazes. The research question addressed in the present article is how data science tools (interpretable mathematical models and machine learning analyses) can be used to automatically identify students’ task-specific strategies from students’ gazes on single histograms. We report on a study of cognitive behavior that uses data science methods to analyze its data. The study consisted of three phases: (1) using a supervised machine learning algorithm (MLA) that provided a baseline for the next step, (2) designing an interpretable mathematical model (IMM), and (3) comparing the results. For the first phase, we used random forest as a classification method implemented in a software package (Wolfram Research Mathematica, ‘Classify Function’) that automates many aspects of the data handling, including creating features and initially choosing the MLA for this classification. The results of the random forests (1) provided a baseline to which we compared the results of our IMM (2). The previous study revealed that students’ horizontal or vertical gaze patterns on the graph area were indicative of most students’ strategies on single histograms. The IMM captures these in a model. The MLA (1) performed well but is a black box. The IMM (2) is transparent, performed well, and is theoretically meaningful. The comparison (3) showed that the MLA and IMM identified the same task-solving strategies. The results allow for the future design of teacher dashboards that report which students use what strategy, or for immediate, personalized feedback during online learning, homework, or massive open online courses (MOOCs) through measuring eye movements, for example, with a webcam.

A human‐centred learning analytics approach for developing contextually scalable K‐12 teacher dashboards

AbstractThis paper describes a Human‐Centred Learning Analytics (HCLA) design approach for developing learning analytics (LA) dashboards for K‐12 classrooms that maintain both contextual relevance and scalability—two goals that are often in competition. Using mixed methods, we collected observational and interview data from teacher partners and assessment data from their students' engagement with the lesson materials. This DBR‐based, human‐centred design process resulted in a dashboard that supported teachers in addressing their students' learning needs. To develop the dashboard features that could support teachers, we found that a design refinement process that drew on the insights of teachers with varying teaching experience, philosophies and teaching contexts strengthened the resulting outcome. The versatile nature of the approach, in terms of student learning outcomes, makes it useful for HCLA design efforts across diverse K‐12 educational contexts. Practitioner notesWhat is already known about this topic Learning analytics that are aligned to both a learning theory and learning design support student learning. LA dashboards that support users to understand the associated learning analytics data provide actionable insight. Design‐based research is a promising methodology for Human‐Centred Learning Analytics design, particularly in the K‐12 educational context. What this paper adds Leveraging a longstanding, yet fluid, research‐practice partnership is an effective design‐based research adaptation for addressing the high variation in instructional practices that characterize K‐12 education. Using both quantitative and qualitative data that reflects students' developing knowledge effectively supports teachers' inquiry into student learning. Teachers' use of learning analytics dashboards is heavily influenced by their perspectives on teaching and learning. Implications for practice and/or policy Impact on student learning outcomes, alongside usability and feasibility, should be included as a necessary metric for the effectiveness of LA design. LA dashboard developers should both leverage learning data that reflect students' developing knowledge and position teachers to take responsive pedagogical action to support student learning. LA researchers and developers should utilize a long‐term, yet fluid, research‐practice partnership to form a multi‐stakeholder, multidisciplinary design team for Human‐Centred Learning Analytics design.

Participatory design of teacher dashboards: navigating the tension between teacher input and theories on teacher professional vision.

In the field of AI in education, there is a movement toward human-centered design in which the primary stakeholders are collaborators in establishing the design and functionality of the AI system (participatory design). Several authors have noted that there is a potential tension in participatory design between involving stakeholders and, thus, increasing uptake of the system on the one hand, and the use of educational theory on the other hand. The goal of the present perspective article is to unpack this tension in more detail, focusing on the example of teacher dashboards. Our contribution to theory is to show that insights from the research field of teacher professional vision can help explain why stakeholder involvement may lead to tension. In particular, we discuss that the sources of information that teachers use in their professional vision, and which data sources could be included on dashboards, might differ with respect to whether they actually relate to student learning or not. Using this difference as a starting point for participatory design could help navigate the aforementioned tension. Subsequently, we describe several implications for practice and research that could help move the field of human centered design further.

Automatic Classification of Online Discussions and Other Learning Traces to Detect Cognitive Presence

Cognitive presence is a core construct of the Community of Inquiry (CoI) framework. It is considered crucial for deep and meaningful online-based learning. CoI-based real-time dashboards visualizing students’ cognitive presence may help instructors to monitor and support students’ learning progress. Such real-time classifiers are often based on the linguistic analysis of the content of posts made by students. It is unclear whether these classifiers could be improved by considering other learning traces, such as files attached to students’ posts. We aimed to develop a German-language cognitive presence classifier that includes linguistic analysis using the Linguistic Inquiry and Word Count (LIWC) tool and other learning traces based on 1,521 manually coded meaningful units from an online-based university course. As learning traces, we included not only the linguistic features from the LIWC tool, but also features such as attaching files to a post, tagging, or using terms from the course glossary. We used the k-nearest neighbor method, a random forest model, and a multilayer perceptron as classifiers. The results showed an accuracy of up to 82% and a Cohen’s κ of 0.76 for the cognitive presence classifier for German posts. Including learning traces did not improve the predictive ability. In conclusion, we developed an automatic classifier for German-language courses based on a linguistic analysis of students’ posts. This classifier is a step toward a teacher dashboard. Our work also provides the first fully CoI-coded German dataset for future research on cognitive presence.

Pear Deck

This article provides a review of Pear Deck which allows teachers to create interactive slide presentations. In this article, we first provide an overview of Pear Decks’ features and some background information about this tool. Then, we discuss its technological features as well as its pedagogical features. Next, we evaluate the teacher fit in terms of theoretical underpinning of this tool and how it can support student learning. We underline that Pear Deck might be beneficial in creating more student-centered and collaborative slide presentations and in allowing teachers provide their students with ongoing formative assessments. We also discuss how Pear Deck can address the needs of diverse group of learners which is crucial in the distant learning environment of COVID-19 pandemic when learners might feel more isolated and might need individualized help. In this respect, we point out that Pear Deck has some features such as Teacher Dashboard which allows teachers to monitor individual student progress and provide immediate tailored feedback. However, we note that some additional features such as student-takeaways, immersive reader and audio options for slides are only available in paid “Premium” version which limits the efficacy of the product. Producer DetailsPear Deck308 E Burlington St #303,Iowa City, IA 52240https://www.peardeck.com

The function of teacher dashboards depends on the amount of time pressure in the classroom situation: Results from teacher interviews and an experimental study

Teacher dashboards are visual displays that provide information to teachers about their learners. In this article, we address teacher dashboards in the context of computer-supported student collaboration in primary education. We examine the role of different types of dashboards for the specific purpose of aiding teachers in identifying which group of collaborating students is in need of support. This question is addressed using qualitative and quantitative approaches. First, an interview study is reported in which teachers’ views (n = 10) on and perceptions of the acceptability of different types of dashboards were examined. Then, the results of an experimental vignette study are reported, which built upon on the interview study, and in which teachers (n = 35) interacted with mirroring or advising dashboards. Together, the studies revealed that the classroom situation, such as differing levels of time pressure, plays an important role regarding what type of dashboard is beneficial for a teacher to use in the classroom. The theoretical contribution of our study lies in a conceptual and empirical investigation of the relation between teachers’ need for control and their perception of different types of dashboards. Our study also points to several practical implications and directions for future research.

Examining teachers’ behavior patterns in and perceptions of using teacher dashboards for facilitating guidance in CSCL

Examining teachers’ behavior patterns in and perceptions of using teacher dashboards for facilitating guidance in CSCL

Using Teacher Dashboards to Assess Group Collaboration in Problem-based Learning

Assessing group collaboration is a critical element in Problem-based Learning (PBL). In asynchronous online PBL settings, facilitators encounter challenges to assess group collaboration because of delayed responses, lack of social cues, and the orchestration load. Teacher dashboards have the potential to support facilitators to assess collaboration by providing synthesized and visualized information about student learning. Previous studies have explored facilitators’ user experience of teacher dashboards. However, little is known about how facilitators with different levels of PBL expertise interpret dashboard information differently. In this study, we analyzed ten PBL facilitators’ utterance moves while interacting with an online teacher dashboard to examine the difference between expert and novice facilitators as they used each visualization. This study can inform the design of teacher dashboards on collaboration assessment.

Connecting a teacher dashboard to a student digital collaborative environment: supporting teacher enactment of problem-based mathematics curriculum

Connecting a teacher dashboard to a student digital collaborative environment: supporting teacher enactment of problem-based mathematics curriculum

How Teacher Characteristics Relate to How Teachers Use Dashboards: Results From Two Case Studies in K-12

Teacher dashboards are a specific form of analytics in which visual displays provide teachers with information about their students; for example, concerning student progress and performance on tasks during lessons or lectures. In the present paper, we focus on the role of teacher dashboards in the context of teacher decision-making in K–12 education. There is large variation in teacher dashboard use in the classroom, which could be explained by teacher characteristics. Therefore, we investigate the role of teacher characteristics — such as experience, age, gender, and self-efficacy — in how teachers use dashboards. More specifically, we present two case studies to understand how diversity in teacher dashboard use is related to teacher characteristics. Surprisingly, in both case studies, teacher characteristics were not associated with dashboard use. Based on our findings, we propose an initial framework to understand what contributes to diversity of dashboard use. This framework might support future research to attribute diversity in dashboard use. This paper should be seen as a first step in examining the role of teacher characteristics in dashboard use in K–12 education.

Using Innovative Methods to Explore the Potential of an Alerting Dashboard for Science Inquiry

Educational technologies, such as teacher dashboards, are being developed to support teachers’ instruction and students’ learning. Specifically, dashboards support teachers in providing the just-in-time instruction needed by students in complex contexts such as science inquiry. In this study, we used the Inq-Blotter teacher-alerting dashboard to investigate whether teacher support elicited by the technology influenced students’ inquiry performance in a science intelligent tutoring system, Inq-ITS. Results indicated that students’ inquiry improved after receiving teachers’ help, elicited by the Inq-Blotter alerts. This inquiry improvement was significantly greater than for matched students who did not receive help from the teacher in response to alerts. Epistemic network analyses were then used to investigate the patterns in the discursive supports provided to students by teachers. These analyses revealed significant differences in the types of support that fostered (versus did not foster) student improvement; differences across teachers were also found. Overall, this study used innovative tools and analyses to understand how teachers use this technological genre of alerting dashboards to dynamically support students in science inquiry.

The equalizing effect of teacher dashboards on feedback in K-12 classrooms

ABSTRACT Teacher dashboards provide real-time information about students’ performance and progress, which help K-12 teachers to adjust feedback to student’ specific needs during learning. Prior research indicated two problems with respect to how teachers provide feedback: (i) teachers do not always select the most effective feedback to support student’ learning and (ii) feedback is not allocated equally to students with different abilities. Specifically, process feedback is considered most effective yet is relatively scarce. In order to understand how dashboards influence the type and allocation of feedback, we compared characteristics of feedback given after dashboard consultation (dashboard-prompted feedback) to feedback triggered by teachers themselves or in response to students’ questions (human-prompted feedback) in thirty-five K-12 classrooms. Results showed that dashboards led to equal amounts of task and process feedback, while human-prompts led to much more task than process feedback and this difference was especially large for low-ability students. Hence the different types of dashboard-prompted feedback were more equally distributed among students of different ability levels. These results indicate that dashboards can have an important equalizing effect on teacher feedback practices.

The Case of a Multiplication Skills Game: Teachers’ Viewpoint on MG’s Dashboard and OSLM Features

Educational games and digital game-based learning (DGBL) provide pupils interactive, engaging, intelligent, and motivating learning environments. According to research, digital games can support students’ learning and enhance their motivation to learn. Given the central role teachers play in the learning process, their perceptions of DGBL play a significant role in the usage and effectiveness of game-based learning. This paper presents the main findings of an online research on primary school teachers’ attitudes toward DGBL. Furthermore, the research investigates teachers’ opinions about the functionalities provided by the implemented Multiplication Game (MG) and the newly incorporated teacher dashboard. The MG is an assessment and skills improvement tool that integrates an adaptation mechanism that identifies student weaknesses on the multiplication tables and in its latest version also supports a strong social parameter. Students can be informed about their own progress as well as the progress of their peers in an effort to examine if social interaction or competition can increase players’ motivation, which is a subject that raised some concerns in the teaching community. The paper describes the functional options offered by the MG dashboard and documents the outcomes of an online survey conducted with the participation of 182 primary school teachers. The survey indicated the potential usefulness of MG and the benefits it can offer as a learning tool to improve pupil multiplication skills and help teachers identify individual pupil skills and difficulties and adapt their teaching accordingly. The analysis applied has found a correlation between teachers’ perceptions about MG and their view on using digital games in general.

Differentiated Teaching With Adaptive Learning Systems and Teacher Dashboards: The Teacher Still Matters Most

Although many schools in the Netherlands have purchased adaptive learning systems (ALSs) to reduce workload and improve differentiated instruction, the use of ALSs with teacher dashboards in the classroom does not in itself necessarily improve differentiated instruction. The question is, what skills and knowledge do teachers need to provide differentiated instruction with the help of ALSs with teacher dashboards and how does this differ from a context in which such technologies are not used? By means of a cognitive task analysis, consisting of classroom observations, interviews, and several expert meetings, teachers' skills, and knowledge necessary for providing differentiated instruction when using teacher dashboards and ALSs were disentangled. These findings were compared with the skills and knowledge needed by teachers providing differentiated instruction without these tools. Results show that teachers' dashboards can support teachers when providing differentiated instruction by providing up-to-date information about students' progress. Nevertheless, coping with the new information provided on the dashboards also requires new skills and knowledge such as interpreting information from the dashboard and understanding how the information in the ALS is calculated. Although there are some differences, the core principles of differentiated instruction-being goal-oriented, continually monitoring, adapting instruction and assignments, being ambitious, stimulating students' self-regulation-are critical for high-quality differentiated instruction regardless of the available tools in the classroom.

A Theoretical and Evidence-Based Conceptual Design of MetaDash: An Intelligent Teacher Dashboard to Support Teachers' Decision Making and Students’ Self-Regulated Learning

Teachers’ ability to self-regulate their own learning is closely related to their competency to enhance self-regulated learning (SRL) in their students. Accordingly, there is emerging research for the design of teacher dashboards that empower instructors by providing access to quantifiable evidence of student performance and SRL processes. Typically, they capture evidence of student learning and performance to be visualized through activity traces (e.g., bar charts showing correct and incorrect response rates, etc.) and SRL data (e.g., eye-tracking on content, log files capturing feature selection, etc.) in order to provide teachers with monitoring and instructional tools. Critics of the current research on dashboards used in conjunction with advanced learning technologies (ALTs) such as simulations, intelligent tutoring systems, and serious games, argue that the state of the field is immature and has 1) focused only on exploratory or proof-of-concept projects, 2) investigated data visualizations of performance metrics or simplistic learning behaviors, and 3) neglected most theoretical aspects of SRL including teachers’ general lack of understanding their’s students’ SRL. Additionally, the work is mostly anecdotal, lacks methodological rigor, and does not collect critical process data (e.g. frequency, duration, timing, or fluctuations of cognitive, affective, metacognitive, and motivational (CAMM) SRL processes) during learning with ALTs used in the classroom. No known research in the areas of learning analytics, teacher dashboards, or teachers’ perceptions of students’ SRL and CAMM engagement has systematically and simultaneously examined the deployment, temporal unfolding, regulation, and impact of all these key processes during complex learning. In this manuscript, we 1) review the current state of ALTs designed using SRL theoretical frameworks and the current state of teacher dashboard design and research, 2) report the important design features and elements within intelligent dashboards that provide teachers with real-time data visualizations of their students’ SRL processes and engagement while using ALTs in classrooms, as revealed from the analysis of surveys and focus groups with teachers, and 3) propose a conceptual system design for integrating reinforcement learning into a teacher dashboard to help guide the utilization of multimodal data collected on students’ and teachers’ CAMM SRL processes during complex learning.

How Teachers integrate Dashboards into their Feedback Practices

In technology empowered classrooms teachers receive real-time data about students' performance and progress on teacher dashboards. Dashboards have the potential to enhance teachers' feedback practices and complement human-prompted feedback that is initiated by teachers themselves or students asking questions. However, such enhancement requires teachers to integrate dashboards into their professional routines. How teachers shift between dashboard- and human-prompted feedback could be indicative of this integration. We therefore examined in 65 K-12 lessons: i) differences between human- and dashboard-prompted feedback; ii) how teachers alternated between human- and dashboard-prompted feedback (distribution patterns); and iii) how these distribution patterns were associated with the given feedback type: task, process, personal, metacognitive, and social feedback. The three sources of feedback resulted in different types of feedback: Teacher-prompted feedback was predominantly personal and student-prompted feedback mostly resulted in task feedback, whereas dashboard-prompted feedback was equally likely to be task, process, or personal feedback. We found two distribution patterns of dashboard-prompted feedback within a lesson: either given in one sequence together (blocked pattern) or alternated with student- and teacher-prompted feedback (mixed pattern). The distribution pattern affected the type of dashboard-prompted feedback given. In blocked patterns, dashboard-prompted feedback was mostly personal, whereas in mixed patterns task feedback was most prevalent. Hence, both sources of feedback instigation as well as the distribution of dashboard-prompted feedback affected the type of feedback given by teachers. Moreover, when teachers advanced the integration of dashboard-prompted feedback in their professional routines as indicated by mixed patterns, more effective types of feedback were given.