Dashboard Design Research Articles

Predict or describe? How learning analytics dashboard design influences motivation and statistics anxiety in an online statistics course.

Based on the achievement goal theory, this experimental study explored the influence of predictive and descriptive learning analytics dashboards on graduate students’ motivation and statistics anxiety in an online graduate-level statistics course. Participants were randomly assigned into one of three groups: (a) predictive dashboard, (b) descriptive dashboard, or (c) control (i.e., no dashboard). Measures of motivation and statistical anxiety were collected in the beginning and the end of the semester via the Motivated Strategies for Learning Questionnaire and Statistical Anxiety Rating Scale. Individual semi-structured interviews were used to understand learners’ perceptions of the course and whether the use of the dashboards influenced the meaning of their learning experiences. Results indicate that, compared to the control group, the predictive dashboard significantly reduced learners’ interpretation anxiety and had an effect on intrinsic goal orientation that depended on learners’ lower or higher initial levels of intrinsic goal orientation. In comparison to the control group, both predictive and descriptive dashboards reduced worth of anxiety (negative attitudes towards statistics) for learners who started the course with higher levels of worth anxiety. Thematic analysis revealed that learners who adopted a more performance-avoidance goal orientation approach demonstrated higher levels of anxiety regardless of the dashboard used.Supplementary InformationThe online version contains supplementary material available at 10.1007/s11423-021-09998-z.

Towards Smart Campus Management: Defining Information Requirements for Decision Making through Dashboard Design

At universities worldwide, the notion of a ‘smart campus’ is becoming increasingly appealing as a response to the multitude of challenges that impact campus development and operation. Smart campus tools are widely used to support students and employees, optimise space use and save energy. Although smart campus tools are supposed to support campus managers in their decision-making processes, the use of the information delivered by smart campus tools and their application in organisational processes has received little attention. In this paper, we focus on the use of dashboards in the connection of IoT information to strategic decision-making processes in the management of university campuses. To this end, we developed a briefing approach for dashboards that expresses the needs of campus management and matches the structure of decision-making processes. In two cases, dashboards based on this approach were use-tested by stakeholders for defining information requirements for IoT applications. The results suggest that users are able to use dashboards for assessing portfolio performance and determining interventions. Through iteration the usability of the dashboard is improved and information requirements are refined, resulting in a brief for a campus management dashboard. The results suggest that the briefing approach can be used to determine IoT information requirements, though further research is required to study indications and contra-indications of the proposed method.

Staying on target: A systematic literature review on learner‐facing learning analytics dashboards

AbstractThe advances in technology to capture and process unprecedented amounts of educational data has boosted the interest in Learning Analytics Dashboard (LAD) applications as a way to provide meaningful visual information to administrators, parents, teachers and learners. Despite the frequent argument that LADs are useful to support target users and their goals to monitor and act upon the information provided, little is known about LADs’ theoretical underpinnings and the alignment (or lack thereof) between LADs intended outcomes and the measures used to evaluate their implementation. However, this knowledge is necessary to illuminate more efficient approaches in the development and implementation of LAD tools. Guided by the self‐regulated learning perspective and using the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) framework, this systematic literature review addressed this gap by examining whether and how learner‐facing LAD’s target outcomes align with the domain measures used to evaluate their implementations. Out of the 1297 papers retrieved from 15 databases, 28 were included in the final quantitative and qualitative analysis. Results suggested an intriguing lack of alignment between LADs’ intended outcomes (mostly cognitive domain) and their evaluation (mostly affective measures). Based on these results and on the premise that LADs are designed to support learners, a critical recommendation from this study is that LADs’ target outcomes should guide the selection of measures used to evaluate the efficacy of these tools. This alignment is critical to enable the construction of more robust guidelines to inform future endeavours in the field. Practitioner notesWhat is already known about this topic There has been an increased interest and investment in learning analytics dashboards to support learners as end‐users. Learner‐facing learning analytics dashboards are designed with different purposes, functionalities and types of data in an attempt to influence learners’ behaviour, achievement and skills. What this paper adds This paper reports trends and opportunities regarding the design of learner‐facing learning analytics dashboards, contexts of implementation, as well as types and features of learner‐facing learning analytics dashboard studies. The paper discusses how affect and motivation have been largely overlooked as target outcomes in learner‐facing learning analytics dashboards. Implications for practice and/or policy Based on the evidence gathered through the review, this paper makes recommendations for theory (eg, inclusion of motivation as an important target outcome). The paper makes recommendations related to the design, implementation and evaluation of learning analytics dashboards. The paper also highlights the need for further integration between learner‐facing learning analytics dashboards and open learner models.

Rigorous Data Validation for Accurate Dashboards: Experience From a Higher Education Institution

Data have become an indispensable aspect of our daily lives. The demand for data visualization tools such as dashboards is driven by the desire to make data—and more importantly, the power of using data to inform decision making—accessible to all. During the current pandemic, the availability of realtime data via various dashboards at the global, national, and local levels empowered many to accurately assess the situation and take appropriate actions, a testament to the value of data visualization. For the same reasons, data dashboards are increasingly popular in higher education to promote data consumption and data-driven decision making. California State University, Fullerton (CSUF) is no exception. CSUF has developed a suite of dashboards in the past few years to promote an operational culture that is rooted in evidence. Discussions about data visualization tools often gravitate towards dashboard design, accessibility, and usability, while neglecting a fundamental (and arguably more critical) issue—the importance of having appropriate and accurate underlying data. The accuracy and adaptability of a dashboard are determined by the accuracy and adaptability of the data behind it, and to ensure such requires a meticulous, streamlined development process. This article is intended to do a “deep dive” into this process.

Data Storytelling with Dashboard: Accelerating Understanding Through Data Visualization in Financial Technology Company Case Study

In the rapid development many organisation rely on context data to support as well as to assist its decision making process. Consequently, Business Intelligence (BI), Dashboard, and Data Visualization emerged as primary tools in early 1990s as a way to help practitioners, data analyst, and data scientist to present context data into an actionable information for decision making process. However, despite its robust and powerful tools, recent study done by Kaggle’s survey in 2017 resulted that in the last five years, many companies were not able to create effective data-driven dashboard due to complex dataset, poor dashboard design, and insufficient storytelling. Hence, understanding of who is going to use dashboard, choosing which data and metrics to visualize in the right context, knowing how to convey information, driving engagement, and persuading audiences are essential in current business practices. This study is aimed to help practitioners to understand the impact of effective dashboard can have on decision making process, to design leveraging dashboard, and to present the dashboard in storytelling. A literature study is performed to gather all relevant information resulted in guidelines for dashboard creator. Case study in financial technology company is applied to experiment and to test the guidelines for assisting dashboard creator to present data-driven insight to the stakeholder.

Establishing a global quality of care benchmark report.

The Movember funded TrueNTH Global Registry (TNGR) aims to improve care by collecting and analysing a consistent dataset to identify variation in disease management, benchmark care delivery in accordance with best practice guidelines and provide this information to those in a position to enact change. We discuss considerations of designing and implementing a quality of care report for TNGR. Eleven working group sessions were held prior to and as reports were being built with representation from clinicians, data managers and investigators contributing to TNGR. The aim of the meetings was to understand current data display approaches, share literature review findings and ideas for innovative approaches. Preferred displays were evaluated with two surveys (survey 1: 5 clinicians and 5 non-clinicians, 83% response rate; survey 2: 17 clinicians and 18 non-clinicians, 93% response rate). Consensus on dashboard design and three data-display preferences were achieved. The dashboard comprised two performance summary charts; one summarising site's relative quality indicator (QI) performance and another to summarise data quality. Binary outcome QIs were presented as funnel plots. Patient-reported outcome measures of function score and the extent to which men were bothered by their symptoms were presented in bubble plots. Time series graphs were seen as providing important information to supplement funnel and bubble plots. R Markdown was selected as the software program principally because of its excellent analytic and graph display capacity, open source licensing model and the large global community sharing program code enhancements. International collaboration in creating and maintaining clinical quality registries has allowed benchmarking of process and outcome measures on a large scale. A registry report system was developed with stakeholder engagement to produce dynamic reports that provide user-specific feedback to 132 participating sites across 13 countries.

Open VPN Application in COVID-19 Pandemic

With the use of COVID-19 information, the way in which COVID-19 prevention and control agencies expand their internal networks is addressed. This article introduces the concept of open VPN technology and puts forward a design scheme using open VPN gateway access, which solves the problem of direct reporting of users accessing the system from the external network. At the same time, the paper uses big data analysis based on the ArcGIS Dashboard design to implement a visual and interactive real-time monitoring system of the global epidemic situation, which can timely and effectively obtain the development of the 2019 coronavirus disease epidemic. The system emphasizes fast and convenient online dynamic display of the global distribution of the epidemic, real-time statistical analysis of the spatial distribution of the epidemic, dynamic display of the severity of the epidemic in various countries and regions and the development trend of the global epidemic, and promotes epidemic monitoring to a certain extent Work development.

A Theoretical and Evidence-Based Conceptual Design of MetaDash: An Intelligent Teacher Dashboard to Support Teachers' Decision Making and Students’ Self-Regulated Learning

Teachers’ ability to self-regulate their own learning is closely related to their competency to enhance self-regulated learning (SRL) in their students. Accordingly, there is emerging research for the design of teacher dashboards that empower instructors by providing access to quantifiable evidence of student performance and SRL processes. Typically, they capture evidence of student learning and performance to be visualized through activity traces (e.g., bar charts showing correct and incorrect response rates, etc.) and SRL data (e.g., eye-tracking on content, log files capturing feature selection, etc.) in order to provide teachers with monitoring and instructional tools. Critics of the current research on dashboards used in conjunction with advanced learning technologies (ALTs) such as simulations, intelligent tutoring systems, and serious games, argue that the state of the field is immature and has 1) focused only on exploratory or proof-of-concept projects, 2) investigated data visualizations of performance metrics or simplistic learning behaviors, and 3) neglected most theoretical aspects of SRL including teachers’ general lack of understanding their’s students’ SRL. Additionally, the work is mostly anecdotal, lacks methodological rigor, and does not collect critical process data (e.g. frequency, duration, timing, or fluctuations of cognitive, affective, metacognitive, and motivational (CAMM) SRL processes) during learning with ALTs used in the classroom. No known research in the areas of learning analytics, teacher dashboards, or teachers’ perceptions of students’ SRL and CAMM engagement has systematically and simultaneously examined the deployment, temporal unfolding, regulation, and impact of all these key processes during complex learning. In this manuscript, we 1) review the current state of ALTs designed using SRL theoretical frameworks and the current state of teacher dashboard design and research, 2) report the important design features and elements within intelligent dashboards that provide teachers with real-time data visualizations of their students’ SRL processes and engagement while using ALTs in classrooms, as revealed from the analysis of surveys and focus groups with teachers, and 3) propose a conceptual system design for integrating reinforcement learning into a teacher dashboard to help guide the utilization of multimodal data collected on students’ and teachers’ CAMM SRL processes during complex learning.

QualDash: Adaptable Generation of Visualisation Dashboards for Healthcare Quality Improvement.

Adapting dashboard design to different contexts of use is an open question in visualisation research. Dashboard designers often seek to strike a balance between dashboard adaptability and ease-of-use, and in hospitals challenges arise from the vast diversity of key metrics, data models and users involved at different organizational levels. In this design study, we present QualDash, a dashboard generation engine that allows for the dynamic configuration and deployment of visualisation dashboards for healthcare quality improvement (QI). We present a rigorous task analysis based on interviews with healthcare professionals, a co-design workshop and a series of one-on-one meetings with front line analysts. From these activities we define a metric card metaphor as a unit of visual analysis in healthcare QI, using this concept as a building block for generating highly adaptable dashboards, and leading to the design of a Metric Specification Structure (MSS). Each MSS is a JSON structure which enables dashboard authors to concisely configure unit-specific variants of a metric card, while offloading common patterns that are shared across cards to be preset by the engine. We reflect on deploying and iterating the design of OualDash in cardiology wards and pediatric intensive care units of five NHS hospitals. Finally, we report evaluation results that demonstrate the adaptability, ease-of-use and usefulness of QualDash in a real-world scenario.

Digital Dashboard for Improving Pharmacy Management: A mixed-method study

Aim: This study aimed to determine the performance indicators required for hospital pharmacies and design a hospital pharmacy dashboard through defining requirements. Methods: This study was conducted in three phases in 2017. A qualitative approach was employed to collect and formulate key performance indicators (KPIs) for hospital pharmacies. A semi-structured questionnaire was constructed to determine KPIs. The data were analyzed using framework analysis. The classic Delphi technique was used to determine the dashboard requirements. Finally, the Dashboard was designed using QlikView12 and implemented in the selected hospital. The data displayed on the pharmacy dashboard was driven by the hospital information system (HIS). Results: The KPIs consisted of three domains including managerial, clinical, and financial indicators. Conclusion: Pharmacy services have been affected by the complexity and diversity of drugs, clinical concerns, and costs. The dashboard can help pharmacies' managers by monitoring. The dashboard design process begins by identifying KPIs that are important to decision-makers.

Increasing resilience via the use of personal data: Lessons from COVID-19 dashboards on data governance for the public good

Abstract Contemporary data tools such as online dashboards have been instrumental in monitoring the spread of the COVID-19 pandemic. These real-time interactive platforms allow citizens to understand the local, regional, and global spread of COVID-19 in a consolidated and intuitive manner. Despite this, little research has been conducted on how citizens respond to the data on the dashboards in terms of the pandemic and data governance issues such as privacy. In this paper, we seek to answer the research question: how can governments use data tools, such as dashboards, to balance the trade-offs between safeguarding public health and protecting data privacy during a public health crisis? This study used surveys and semi-structured interviews to understand the perspectives of the developers and users of COVID-19 dashboards in Hong Kong. A typology was also developed to assess how Hong Kong’s dashboards navigated trade-offs between data disclosure and privacy at a time of crisis compared to dashboards in other jurisdictions. Results reveal that two key factors were present in the design and improvement of COVID-19 dashboards in Hong Kong: informed actions based on open COVID-19 case data, and significant public trust built on data transparency. Finally, this study argues that norms surrounding reporting on COVID-19 cases, as well as cases for future pandemics, should be co-constructed among citizens and governments so that policies founded on such norms can be acknowledged as salient, credible, and legitimate.

TaBAT: Design and Experimentation of a Learning Analysis Dashboard for Teachers and Learners

Aim/Purpose: Since the beginning of the COVID-19 pandemic, many countries have adopted online education as an alternative to face-to-face courses. This has increased awareness of the importance of analyzing learning data left by students to improve and evaluate the learning process. This article presents a new tool, named TaBAT, created to work with different LMSs in the form of dashboards accessible online and allowing teachers to monitor the progress of their learners and at the same time allow learners to visualize their learning process. Background: TaBAT is designed based on the results of our previous research on factors that can influence the success of online learners, where we proposed and statistically validated a model for assessing the success of online learners called e-LSAM (e-Learner Success Assessment Model). Methodology: Two studies are presented in this article. The first one is conducted on a group of students from two classes (each composed of two groups) of a higher institute in Morocco, who participated in courses organized in blended learning on the Moodle platform. For each class, one of the two groups had access to the experiment to use the TaBAT tool (exposed group) to analyze the learning traces, while the second group did not have access to the dashboard (control group). The second study aimed to understand the impact of the usage of the TaBAT tool on the two exposed groups. Contribution: The purpose of this article is to present a new analysis tool as well as to test this tool and to evaluate its impact on self-regulation and the prediction of academic success and, finally, to see how these students evaluate this tool. Findings: The results of the TaBAT usage demonstrate the effectiveness of the success algorithm, based on our theoretical model e-LSAM. The results also prove that this tool was able to increase the performance of the students of both groups exposed. The general evaluations of the participants also confirmed these results. Impact on Society: The article proposes a tool for institutions to facilitate the monitoring and control of students’ learning process. The tool provides visual information for teachers to study and react to in the educational context and gives students visualizations to promote their self-reflection and increase their performance and academic success. Future Research: Generalize the use of the TaBAT tool, incorporating both private and public institutions, in order to confirm the results obtained in this article and at the same time improve the self-regulation and academic success of learners.

Evaluation of innovative solutions for e-mobility

This paper presents a methodology for the evaluation of innovative solutions designed by the GreenCharge project to advance electric mobility. The methodology is based on the CIVITAS evaluation framework and it is specialised according to the GreenCharge requirements. In particular, this paper provides an extensive list of Key Performance Indicators (KPIs) and which of them will be evaluated in three pilots to provide a quantitative estimation of impact of technology innovation. Evaluation will use automatic computation of data collected in pilots, simulation techniques and analysis of surveys and interviews. The paper describes how to deliver and how to present evaluation results. The detailed schedule of data collection and evaluation activities has been planned in collaboration with implementation activities. Finally, a preliminary requirement analysis of simulation and a dashboard design for KPIs presentation are discussed.

Optimizing the mutual arrangement of pilot indicators on an aircraft dashboard and analysis of this procedure from the viewpoint of quantum representations

The purpose of this work is to present the first attempt to provide quantitative analysis and objective justification for designers' decisions that relate to the arrangement of pilot indicators on an aircraft dashboard with the use of video oculography measurements. To date, such decisions have been made only based on the practical experience accumulated by designers and subjective expert assessments. A new method for optimizing the mutual arrangement of the dashboard indicators is under consideration. This is based on iterative correction of the gaze transition probability matrix between the selected zones of attention, to minimize the difference between the stationary distribution of relative frequencies of gaze that are staying in these zones and the corresponding desirable target eye movements that are given for distribution for qualified pilots. When solving the subsequent multidimensional scaling problem, the gaze transition probability matrix that is obtained is considered to be the similarity matrix, the elements of which quantitatively characterize the proximity between the zones of attention. The main findings of this novel work are as follows: the use of oculography data to justify dashboard design decisions, the optimizing method itself, and its mathematical components, as well as analysis of the optimization in question from the viewpoint of quantum representations, all revealed design mistakes. The results that were obtained can be applied for prototyping variants of aircraft dashboards by rearranging the display areas associated with the corresponding zones of attention.

Dashboard Visualization of Information for Emergency Medical Services.

A literature search was conducted in PubMed, IEEE and ACM. The goal was to identify design principles for dashboards. Additionally, several interviews and meetings were held with the EMS staff of the Emmental Hospital and with those of another hospital. The aim was to identify requirements of the EMS staff towards such an organizational dashboard and to transform them into use cases. Considering the collected requirements and standards of dashboard design, a prototype of a dashboard was developed. It consists of several modules that show relevant information items such as news or traffic information. Due to this modular development, content is easily interchangeable. The most important information for the EMS is shown on the dashboard aiming at saving time for information gathering. A digital dashboard offers many advantages and optimization possibilities compared to an analog whiteboard. For example, such a dashboard can be connected to other systems and data can be automatically included. Although we developed our dashboard in cooperation with the EMS of a specific hospital, it can easily be applied and adjusted to other EMS. As a next step, we will perform usability tests with the prototype and start implementing the dashboard.

Responsible, Automated Data Gathering for Timely Citizen Dashboard Provision During a Global Pandemic (COVID-19)

Creating a public understanding of the dynamics of a pandemic, such as COVID-19, is vital for introducing restrictive regulations. Gathering diverse data responsibly and sharing it with experts and citizens in a timely manner is challenging. This article reviews methodologies of COVID-19 dashboard design and discusses both technical and non-technical challenges associated. Advice and lessons learned from building a citizen-focused, automated county-precision dashboard for Germany are shared. Within four months, the web-based tool had 5 million unique visitors and 70 million sessions. Three developers set up the basic version in less than one week. Early on, data was screen scraped. An iterative process improved timeliness by adding more fine-grained data sources. A collaborative online table editor enabled near real-time corrections. Alerting was setup for errors, and statistics apply for sanity checking. Static site generation and a content delivery network help to serve large user loads in a timely manner. The flexible design allowed to iteratively integrate more complex statistics based on expert knowledge built on top of the collected data and secondary data sources such as ICU beds and citizen movement.

Self-regulation and emotion matter: A case study of instructor interactions with a learning analytics dashboard

Learning analytics (LA) is providing new methodologies that are being applied to the design and application of dashboards to support teaching and learning. However, few studies attempt to understand how instructors interact with an LA dashboard and how self-regulated learning (SRL) activities and emotions of instructors occur and co-occur in the interaction. The current study investigates ten instructors’ SRL activities and epistemic emotions by analyzing the screen capture videos and think-aloud data while they interact with an LA dashboard designed to support the online asynchronous collaboration of multiple groups. The results reveal that instructors demonstrated two ways of navigating LA dashboards, and they relied heavily on the conversation explorer feature. Instructors were mostly engaged in elaboration, monitoring, and evaluation activities and they frequently experienced confusion and enjoyment. Expert instructors were more likely to refer to their personal teaching experience and demonstrated more epistemic emotions than novice instructors. This study contributes to the literature on SRL and teacher emotions by revealing the critical role of elaboration, monitoring, evaluation, and epistemic emotions when instructors attempt to understand a LA dashboard by themselves. These findings highlight the importance of providing pedagogical assistance to teachers who are trying to navigate between group dynamics and visualizations viewed using LA dashboards.

An Actionable Orchestration Dashboard to Enhance Collaboration in the Classroom

The orchestration of collaborative learning activities in technology-enhanced classrooms has become a nontrivial endeavor for educators. Depending on the behaviors and needs of students that emerge in real educational situations, educators may need to orchestrate activity adaptations on the fly. These adaptations may range from the provision of additional scaffolding by the educator (e.g., the educator's participation in a group discussion) to a change in the planned pedagogical scenario (e.g., the duration). This article aims to contribute to the orchestration of technology-mediated collaborative learning sessions in a classroom context. We present the design, implementation, and evaluation of a teacher-facing dashboard that supports teachers in orchestrating scripted collaboration. Evaluation studies were conducted in 16 classroom sessions. The findings indicate that teachers found the information on the dashboard to be actionable and help facilitate just in time support to student groups.

A survey on automatic infographics and visualization recommendations

Automatic infographics generators employ machine learning algorithms/user-defined rules and visual embellishments into the creation of infographics. It is an emerging topic in the field of information visualization that has requirements in many sectors, such as dashboard design, data analysis, and visualization recommendation. The growing popularity of visual analytics in recent years brings increased attention to automatic infographics. This creates the need for a broad survey that reviews and assesses the significant advances in this field. Automatic tools aim to lower the barrier for visually analyzing data by automatically generating visualizations for analysts to search and make a choice, instead of manually specifying. This survey reviews and classifies automatic tools and papers of visualization recommendations into a set of application categories including network-graph visualizations, annotation visualizations, and storytelling visualization. More importantly, this report presents several challenges and promising directions for future work in the field of automatic infographics and visualization recommendations.

Enabling Managerial Decision Making: The Use of Multiple Views in Dashboards

A very useful way to enable decision making among overloaded managers is to represent data visually in graphs, trends, bar charts, maps etc. Graphical representations are superior to textual information when it comes to trends, correlation and other quantifiable information but only if the encoding of such information is both appropriate to the data, and uses an optimal dashboard design. Several review studies find that the management literature has failed to keep pace with the developments of dashboards, and that no agreement exists on how exactly a dashboard should contain (e.g., number of views) and what it should do. Thus, to apply visual representations efficiently and effectively still represent key challenges for creating value and enabling dashboard use in B2B settings. In this paper we experimentally test the efficiency, effectiveness, subjective preference among participants, and physiological gaze fixation count/time across two dashboard settings, one with three-view inserted and one with nine-view inserted. We use questions as well as eye-tracking measures. We find effects pointing to superiority of the nine-view dashboards for some tasks while the three-view dashboard performs better on other tasks. We discuss the implication of these findings in managerial decision making.