Workload Measurement Research Articles

Comparison of weighted-full time equivalent (W-FTE), conventional FTE, and NASA TLX in mental workload analysis (a case study)

Human resources is one of the company’s most important resources. The optimum performance of each employee will undoubtedly be achieved when the workload given to each employee is evenly distributed. Therefore, a workload measurement is required to identify whether each employee’s given workload is evenly distributed. XYZ Inc. is a steel manufacturing company. The production process for each order can take about 1-3 months. Especially for production in the Pressure Part, it can take a long time due to the product’s shape, massive volume, and complexity. In this research, workload measurements were conducted with employees (n=7) of the Production Planning Control of the Pressure Part using a combination of weighted FTE, conventional FTE, and NASA-TLX. The obtained index value results using the FTE is 1.69 (overload), and using W-FTE is 2,3 (overload). The result of the NASA-TLX method is an average WWL value of 76.19 (high). Index of W-FTE is after the FTE index value has been determined, the optimal number of employees is calculated based on the FTE index. Based on the results, adding three new employees (n=10) to equalize the workload in production planning control is necessary to lower the average FTE index value by 1.18 (inload).

Do players with superior physiological attributes outwork their less-conditioned counterparts? A study in Gaelic football.

This study investigated the association of physiological attributes with in-game workload measures during competitive Gaelic football match-play. Fifty-two male developmental level Gaelic football players (mean ± SD; age: 22.9 ± 3.8 years) underwent measurements of anthropometric characteristics, running speed, muscular strength and power, blood lactate (BLa), running economy and aerobic capacity during two separate testing visits. Global Positioning System units (18-Hz) were used to record players in-game workloads during a competitive match 1-week following the baseline physiological assessments. Results indicated that players body fat percentage, drop jump height (DJ) and running velocity at 4 mmol · L-1 BLa were significantly associated with the number of high-speed runs completed (Adjusted R2 26.8% to 39.5%; p < 0.05) while 20 m running speed, running velocity at 2 mmol · L-1 BLa and DJ were significantly associated with the number of accelerations completed (Adjusted R2 17.2% to 22.0%; p < 0.05) during match-play. Additionally, aerobic capacity and body fat percentage were significantly associated with total distance (Adjusted R2 14.4% to 22.4%; p < 0.05) while body fat percentage, DJ and 20 m running speed were significantly associated with high-speed distance (Adjusted R2 17.8% to 22.0%; p < 0.05). Players were also divided into higher-standard and lower-standard groups using a median split of these physiological attributes. Players in the higher-standard groups completed significantly more high-speed runs and accelerations and covered significantly larger total and high-speed distances (+10.4% to +36.8%; ES = 0.67 to 0.88; p < 0.05) when compared to the lower-standard groups. This study demonstrates that superior levels of physical conditioning are associated with larger in-game workloads during Gaelic football match-play.

The Relationship Between Workload and Competency on Employee Commitment

This study examines the individual and combined effects of workload and competency on employee commitment within the Business Unit of Building a Prosperous Family Economy in Bandung Regional 1 (PNM Cimahi Branch). Workload and competency are pivotal factors that can significantly influence employee commitment, particularly in demanding work environments. A quantitative approach was employed, gathering primary data via surveys from 248 employees at the PNM Cimahi Branch. The collected data included measurements of employee workload, competency levels, and commitment to the organization. Path analysis was utilized to assess the relationships between these variables. The findings indicate that both workload and competency positively influence employee commitment, both individually and when considered together.

Pengukuran Beban Kerja Psikologis Operator Di Bay 4 (Preparation) PT XYZ Unit Usaha Passenger Boarding Bridge Menggunakan Metode NASA-TLX

Psychological workload is a workload that is felt when doing work that is formed cognitively, so that the workload is felt by each worker. As a result of these work activities will have an impact on worker productivity so it is necessary for someone to know how much workload is felt on the work activities that are being carried out. Subjective workload measurement can be done using one method, namely the NASA-TLX method. This physical and mental activity has a consequence that is the emergence of workload. Bay 4 has a very important role in PT. XYZ Passenger Boarding Bridge Business Unit which has the task of making the initial components, for all workers. The psychological workload of operators here is expected to be low so that all processes can run smoothly so that production targets can be achieved. Based on the results of research conducted using the NASA-TLX method, it can be concluded that overall the workload of workers working in Bay 4 is in the high category with an average WWL value of 66.02. This is one that must be dealt with quickly so that the work in Bay 4 can go as expected.

Analisis Beban Kerja di Laboratorium Produksi PT. XYZ Dengan Metode CVL dan NASA-TLX

To evaluate the workload, one can consider the combination of physical and mental effort that employees undergo. The quantification of physical workload is accomplished through the utilization of the Cardiovascular Load (%CVL) methodology, whereas the evaluation of mental workload is conducted employing the NASA-TLX methodology. Pulse measurement is employed to gather data on the physical exertion level. Concurrently, the cognitive load was alleviated by administering NASA-TLX surveys to the personnel. The physical workload measurements were acquired from a sample of 12 employees who showed no indications of fatigue. Meanwhile, the measurements of mental workload indicate a substantial level of both elevated and extremely elevated workload One employee was classified as having an extremely heavy workload, while the majority of respondents (11 out of 12) assessed their workload as high. The results suggest that around 33% of employees face a substantial mental workload, underscoring the need for a leadership policy focused on workload reduction.

Authoring Moving Parts of Objects in AR, VR and the Desktop

Creating digital twins of real objects is becoming more popular, with smartphones providing 3D scanning capabilities. Adding semantics to the reconstructed virtual objects is important to possibly reproduce training scenarios that otherwise could demand significant resources or happen in dangerous scenarios in some cases. The aim of this work is to evaluate the usability of authoring object component behaviors in immersive and non-immersive approaches. Therefore, we present an evaluation of the perceived ease of use to author moving parts of objects under three different conditions: desktop, augmented reality (AR) and virtual reality (VR). This paper provides insights into the perceived benefits and issues that domain experts might encounter when authoring geometrical component behaviors across each interface. A within-subject study is the major contribution of this work, from which is presented an analysis based on the usability, workload and user interface preferences of participants in the study. To reduce confounding variables in the study, we ensured that the virtual objects and the environment used for the evaluation were digital twins of the real objects and the environment that the experiment took place in. Results show that the desktop interface was perceived as more efficient and easier to use based on usability and workload measures. The desktop was preferred for performing component selection but no difference was found in the preference for defining a behavior and visualizing it. Based on these results, a set of recommendations and future directions are provided to achieve a more usable, immersive authoring experience.

Evaluation of physiological responses to mental workload in n-back and arithmetic tasks

Working memory tasks, such as n-back and arithmetic tasks, are frequently used in studying mental workload. The present study investigated and compared the sensitivity of several physiological measures at three levels of difficulty of n-back and arithmetic tasks. The results showed significant differences in fixation duration and pupil diameter among three task difficulty levels for both n-back and arithmetic tasks. Pupil diameters increase with increasing mental workload, whereas fixation duration decreases. Blink duration and heart rate (HR) were significantly increased as task difficulty increased in the n-back task, while root mean square of successive differences (RMSSD) and standard deviation of R-R intervals (SDNN) were significantly decreased in the arithmetic task. On the other hand, blink rate and Galvanic Skin Response (GSR) were not sensitive enough to assess the differences in task difficulty for both tasks. All significant physiological measures yielded significant differences between low and high task difficulty except for SDNN. Practitioner summary: This study aimed to assess the sensitivity levels of several physiological measures of mental workload in n-back and arithmetic tasks. It showed that pupil diameter was the most sensitive in both tasks. This study also found that most physiological indices are sensitive to an extreme change in task difficulty levels.

Cutting the Gordian (workload) Knot? Adding data services to academic library public services

The data librarianship as a profession produces on workloads can't give us the answers we seek. The library and information science literature wants two irreconcilable things out of its workload data: 1) aggregate comparable data to document and measure use of libraries and its value; and 2) accurate descriptions to document and measure the individual work done by librarians. That is our Gordian Knot. We propose here to change the question asked: how can we achieve a reasonable balance of workload within a group of librarians? That of course implies a focus on a library of a specific type: here a medium-sized academic library of an R2 institution. The goal was to answer a common and longstanding question: we are in continual process of assessing what needs to be done and how/where to shift workloads, but how do we know we're doing it in a reasonable and fair way beyond anecdotes and intuitions? We developed a weighted measure of public services workload in order to assess and track and assign a) areas of declining workload, b) areas of increasing workload (data services), and c) a balance between library divisions contributing to public services.

Investigation of eye movement measures of mental workload in healthcare: Can pupil dilations reflect fatigue or overload when it comes to health information system use?

The use of health information systems (HIS) can result in high workloads and, consequently, poor performance characterized by e.g. increased occurrence of errors among clinicians. Pupillometry offers a good possibility to measure mental workload in a dynamic work setting. Currently, there is a lack of empirical research in the context of healthcare settings. Therefore, the aim of the present study was to examine whether specific eye movement measures are suitable for measuring mental workload in the healthcare setting, especially when working with HIS. 49 persons participated in our simulation-lab study. They had to complete a system-related task as well as an increasing n-back task. Both tasks were modified regarding task difficulty. Results show significant differences for objective and subjective workload measures between increasing task levels. There are also hints for an overload/fatigue indicator in pupil data. Our results are limited in terms of external validity, causality and effects. Future work should focus on high-fidelity simulations and less time-consuming analysis approaches.

11 NASA-TLX Workload Profile of the Trail Making Test

Objective:Workload is a useful construct in human factors and neuroergonomics research that describes “the perceived relationship between the amount of mental [and physical] processing capability or resources and the amount required by the task”. We apply this concept to neuropsychology and assess several dimensions of workload as it relates to performance on the Trail Making Test.Participants and Methods:Twenty college students completed the Trail Making Test (TMT). After completion of each Part A and B, workload was assessed with the NASA-Task Load Index (NASA-TLX), a popular self-report measure of workload including subscales: Mental Demand, Physical Demand, Temporal Demand, Performance, Effort, and Frustration, with an overall average total score as well.Results:Completion time differed of course between Parts A and B (p &lt; .001). Of more interest, overall workload differed between TMT A (M = 20.33, SD = 13.32) and TMT B (M = 35.79, SD = 17.37) (p &lt; .001, h2 = .68). The greatest subscale differences were with Mental Demand (p &lt; .001, h2 = .68) and Effort (p &lt; .001, h2 = .59), but Physical Demand also showed a difference (p &lt; .007, h2 = .33). Temporal Demand showed the smallest and nonsignificant difference (p = .081, h2 = .152).Conclusions:Based on previous research in our lab, most results were expected and understandable. As we know with the TMT, Part B is more cognitively demanding (in various ways) than Part A. The greater Physical Demand with Part B is a somewhat more complex finding, needing a solid explanation. Finally, the NASA-TLX appears to be a valid instrument of workload with a standard neuropsychologist test. We argue it can provide useful interesting information in the assessment of cognitive status in clinical populations.

On-Call Workload Differences in Neurosurgery: Resident Call "Weather" and a Departure from Colored Cloud Labels.

Labeling residents as "black" or "white" clouds based on perceived or presumed workloads is a timeworn custom across medical training and practice. Previous studies examining whether such perceptions align with objective workload patterns have offered conflicting results. We assessed whether such peer-assigned labels were associated with between-resident differences in objective, on-call workload metrics in three classes of neurosurgery junior residents. In doing so, we introduce more inclusive terminology for perceived differences in workload metrics. Residents were instructed to complete surveys to identify "sunny", "neutral", and "stormy" residents, reflecting least to greatest perceived workloads, of their respective classes. We retrospectively reviewed department and electronic medical records to record volume of on-call work over the first 4 months of each resident's 2nd postgraduate academic year. Inter-rater agreement of survey responses was measured using Fleiss' kappa. All statistical analyses were performed with a significance threshold of P < .05. Across all classes, there was strong inter-rater agreement in the identification of stormy and sunny residents (Kappa = 1.000, P = .003). While differences in on-call workload measures existed within each class, "weather" designations did not consistently reflect these differences. There were significant intraclass differences in per shift consult volume in two classes ( P = .035 and P = .009); however, consult volume corresponded to a resident's weather designations in only one class. Stormy residents generally saw more emergencies and, in 2 classes, performed more bedside procedures than their peers. Significant differences in objective on-call experience exist between junior neurosurgery residents. Self- and peer-assigned weather labels did not consistently align with a pattern of these differences, suggesting that other factors contribute to such labels.

Toward Workload-Based Adaptive Automation: The Utility of fNIRS for Measuring Load in Multiple Resources in the Brain

We investigate the utility of functional near-infrared spectroscopy (fNIRS) for workload-based adaptive automation through the lens of multiple resource theory. We focus on the criteria of unobtrusiveness, responsiveness, load sensitivity (low vs high load), and load diagnosticity (differentiating types of load). We report a large meta-review, in which we conclude that only a few studies were suitable for evaluating sensitivity and diagnosticity in complex real-world tasks. While these reveal that the fNIRS signal is adequately sensitive to gradations of load level changes (sensitivity), the diagnosticity of fNIRS to different sources of cognitive load remained uncertain. We manipulated mental load of a complex shape sorting task via working memory load (WM) and visual perceptual load (VL), while a secondary auditory task was present throughout. We measured the effect of these manipulations at the group-level using conventional secondary and eyetracking workload measures, as well as hemodynamic response in specific functional regions in the brain, including regions involved in multi-tasking (MT), VL, WM, and auditory load (AL). Our findings revealed that fNIRS is both sensitive and diagnostic to load in complex tasks, with greater sensitivity revealed by deoxyhemoglobin than oxyhemoglobin and the brain regions associated with diagnosticity align with neuroscience literature on perceptual load, WM, and goal-directed multitasking.

Event-related potentials reveal visual episodic memory deficits in patients with temporal lobe epilepsy

ObjectiveTemporal lobe epilepsy (TLE) patients usually suffer from impaired episodic memory (EM), but its underlying electrophysiologic mechanism and impacted cognitive performance are unclear. We aim to investigate the association between episodic memory reserve and physiological measures of memory workload in TLE patients using Event-related potentials (ERP). MethodsA change detection task with image stimuli assesses visual episodic memory. During the memory encoding and decoding phases, the ERP signals were analyzed from twenty-nine TLE patients (twelve with left TLE patients, seventeen with TLE), and thirty healthy controls. Given that EM is a complex process involving many fundamental cognitive processes, the amplitudes and latencies of EM-related ERP (FN400, late positive potential (LPC), and late posterior negativity (LPN)), and the ERP reflecting the fundamental processes (P100, N100, P200, and P300) were calculated. Then we used a three-by-two factorial design on the ERP metrics for interaction and main effects. The correlation analysis among Wechsler Memory Scales-Chinese Revision (WMS-RC) results, behavioral data, and the ERPs was carried out. ResultsThe TLE patients performed worse in WMS-RC and the memory task. The increased P200 and decreased P300 amplitudes were observed in the TLE patients, and LPN was abnormal in only LTLE patients. For EM-related components, differences were observed in both the LTLE and RTLE patients: the lack of the FN400 effect, the lack of the reversed LPC effect, and the reduced FN400. No significant inter-group difference was detected for the latencies of all the ERPs. Additionally, there were significant correlations among WMS-RC scores, behaviors, and some ERP amplitudes. ConclusionsThe impaired EM is linked to the increased P200 and decreased P300 amplitudes. LPN seems to be sensitive to left temporal lobe dysfunction. More importantly, the abnormal old or new effects of the FN400 and LPC, and the reduced FN400 amplitude might be associated with the visual EM deficit in the TLE patients. These findings may assist in the deep understanding of the EM disorder and the evaluation of the side effects of antiepileptic drugs.

Measuring Emergency Department Workload Perception Using Electronic Medical Record Measures of Patient Volume and Acuity

BackgroundWorkload in the emergency department (ED) fluctuates and there is no established model for measurement of clinician-level ED workload. ObjectiveThe aim of this study was to measure perceived ED workload and assess the relationship between perceived workload and objective measures of workload from the electronic medical record (EMR). MethodsThis study was conducted at a tertiary care, academic ED from July 1, 2020 through April 13, 2021. Attending workload perceptions were collected using a 5-point scale in three care areas with variable acuity. We collected eight EMR measures thought to correlate with perceived workload. EMR values were compared across areas of the department using ANOVA and correlated with attending workload ratings using linear regression. ResultsWe collected 315 unique workload ratings, which were normally distributed. For the entire department, there was a weak positive correlation between reported workload perception and mean percentage of inpatient admissions (r = 0.23; p < 0.001), intensive care unit admissions (r = 0.2; p < 0.001), patient arrivals per shift (r = 0.14; p = 0.017), critical care billed visits (r = 0.22; p < 0.001), cardiopulmonary resuscitation code activations (r = 0.2; p < 0.001), and level 5 visits (r = 0.13; p = 0.02). There was weak negative correlation for ED discharges (r = –0.23; p < 0.001). Several correlations were stronger in individual care areas, including percent admissions in the lowest-acuity area (r = 0.43; p = 0.033) and patient arrivals in the highest-acuity area (r = 0.44; p < .01). No significant correlation was found in any area for observation admissions or trauma activations. ConclusionsIn this study, EMR measures of workload were not closely correlated with ED attending physician workload perception. Future study should examine additional factors contributing to physician workload outside of the EMR.

The use of electronic health record embedded MRC-ICU as a metric for critical care pharmacist workload.

A lack of pharmacist-specific risk-stratification scores in the electronic health record (EHR) may limit resource optimization. The medication regimen complexity-intensive care unit (MRC-ICU) score was implemented into our center's EHR for use by clinical pharmacists. The purpose of this evaluation was to evaluate MRC-ICU as a predictor of pharmacist workload and to assess its potential as an additional dimension to traditional workload measures. Data were abstracted from the EHR on adult ICU patients, including MRC-ICU scores and 2 traditional measures of pharmacist workload: numbers of medication orders verified and interventions logged. This was a single-center study of an EHR-integrated MRC-ICU tool. The primary outcome was the association of MRC-ICU with institutional metrics of pharmacist workload. Associations were assessed using the initial 24-h maximum MRC-ICU score's Pearson's correlation with overall admission workload and the day-to-day association using generalized linear mixed-effects modeling. A total of 1205 patients over 5083 patient-days were evaluated. Baseline MRC-ICU was correlated with both cumulative order volume (Spearman's rho 0.41, P < .001) and cumulative interventions placed (Spearman's rho 0.27, P < .001). A 1-point increase in maximum daily MRC-ICU was associated with a 31% increase in order volume (95% CI, 24%-38%) and 4% increase in interventions (95% CI, 2%-5%). The MRC-ICU is a validated score that has been previously correlated with important patient-centered outcomes. Here, MRC-ICU was modestly associated with 2 traditional objective measures of pharmacist workload, including orders verified and interventions placed, which is an important step for its use as a tool for resource utilization needs.

Eye-Tracking in Physical Human-Robot Interaction: Mental Workload and Performance Prediction.

In Physical Human-Robot Interaction (pHRI), the need to learn the robot's motor-control dynamics is associated with increased cognitive load. Eye-tracking metrics can help understand the dynamics of fluctuating mental workload over the course of learning. The aim of this study was to test eye-tracking measures' sensitivity and reliability to variations in task difficulty, as well as their performance-prediction capability, in physical human-robot collaboration tasks involving an industrial robot for object comanipulation. Participants (9M, 9F) learned to coperform a virtual pick-and-place task with a bimanual robot over multiple trials. Joint stiffness of the robot was manipulated to increase motor-coordination demands. The psychometric properties of eye-tracking measures and their ability to predict performance was investigated. Stationary Gaze Entropy and pupil diameter were the most reliable and sensitive measures of workload associated with changes in task difficulty and learning. Increased task difficulty was more likely to result in a robot-monitoring strategy. Eye-tracking measures were able to predict the occurrence of success or failure in each trial with 70% sensitivity and 71% accuracy. The sensitivity and reliability of eye-tracking measures was acceptable, although values were lower than those observed in cognitive domains. Measures of gaze behaviors indicative of visual monitoring strategies were most sensitive to task difficulty manipulations, and should be explored further for the pHRI domain where motor-control and internal-model formation will likely be strong contributors to workload. Future collaborative robots can adapt to human cognitive state and skill-level measured using eye-tracking measures of workload and visual attention.

Hearing loss impacts aviator performance and cognitive workload during simulated flight

Hearing loss can render an aviator more susceptible to the adverse effects of degraded communication signals and consequently lead to an increased allocation of mental resources to hear (referred to as listening effort). Army aviation hearing standards, which are primarily based on pure tone and speech recognition test scores in quiet environments, do not necessarily predict the functional impact of hearing loss. The Army has recently adopted a new Military Operational Hearing Test (MOHT) to assess the functional impact of hearing loss. The current study aimed to validate the MOHT, specifically for aviators, and measure subjective workload, task performance, and physiological workload across different simulated hearing loss conditions in both clinical and flight simulator settings. Participants were Army aviators with normal hearing thresholds and underwent clinical testing using the MOHT for normal and hearing loss conditions. In addition to audiometric data, flight performance data were collected from pilots operating a full-motion UH-60 Black Hawk flight simulator for different levels of workloads and compared across the different hearing loss conditions. Subjective and physiological measurements of cognitive workload during simulated flight were also collected. Findings from this study will be leveraged in developing future protocols for aeromedical standards and evaluating hearing loss mitigation strategies using various headset technologies.

HUBUNGAN BEBAN KERJA DENGAN KINERJA TENAGA KEPERAWATAN

Workload is an important concern for the nursing profession which will have an impact on the performance of nursing staff in providing nursing services. This study aims to determine the results of measuring workload and performance of nursing staff in the Outpatient Installation of Dumai City Regional Hospital. The research method was quantitative with a cross-sectional study design. The research was conducted at the Dumai City Hospital Outpatient Installation from June-July 2023. The population of this study was 33 people, the sample used was 33 respondents taken using the total sampling technique. Workload measurement is carried out using the work sampling method through observation. Performance is obtained using secondary data. The data analysis used was univariate analysis and bivariate analysis with Pearson correlation, chi-square and linear regression on work sampling variables only. The research results showed that the calculation of heavy workload in orthopedic and neurosurgery clinics was 93.75%. With direct productive activities in wound care (19.23%) and indirect productive activities in completing the SPPK (Proof of Health Services) (13.01%). Nursing staff with a heavy workload was 27.28%, and a light workload was 72.72%. Nursing staff with good performance was 75.8% and very good 24.2%. There is no relationship between workload and the performance of nursing staff (p-value= 1.000, OR=0.750). Research Ethics Pass Letter Number: 145/KEPK/UHTP/VI/2023 dated June 19 2023. Evaluation and restructuring of work systems include reducing unproductive workloads, adjusting working hours, or allocating resources more efficiently and training and competency development of nursing staff.

This Is Your Brain on Autopilot 2.0: The Influence of Practice on Driver Workload and Engagement During On-Road, Partially Automated Driving.

This on-road study employed behavioral and neurophysiological measurement techniques to assess the influence of six weeks of practice driving a Level 2 partially automated vehicle on driver workload and engagement. Level 2 partial automation requires a driver to maintain supervisory control of the vehicle to detect "edge cases" that the automation is not equipped to handle. There is mixed evidence regarding whether drivers can do so effectively. There is also an open question regarding how practice and familiarity with automation influence driver cognitive states over time. Behavioral and neurophysiological measures of driver workload and visual engagement were recorded from 30 participants at two testing sessions-with a six-week familiarization period in-between. At both testing sessions, participants drove a vehicle with partial automation engaged (Level 2) and not engaged (Level 0) on two interstate highways while reaction times to the detection response task (DRT) and neurophysiological (EEG) metrics of frontal theta and parietal alpha were recorded. DRT results demonstrated that partially automated driving placed more cognitive load on drivers than manual driving and six weeks of practice decreased driver workload-though only when the driving environment was relatively simple. EEG metrics of frontal theta and parietal alpha showed null effects of partial automation. Driver workload was influenced by level of automation, specific highway characteristics, and by practice over time, but only on a behavioral level and not on a neural level. These findings expand our understanding of the influence of practice on driver cognitive states under Level 2 partial automation.

Can a code-based approach to marking and feedback reduce teachers’ workload? An evaluation of the FLASH marking intervention

ABSTRACT Teacher workload is an important policy concern in many education systems around the world, often considered a contributory factor in teacher attrition. One aspect of workload that could be addressed is reducing the amount of written marking and feedback that teachers do. This article reports on the results of an evaluation of FLASH Marking, an intervention aimed at reducing teachers’ marking workload. FLASH Marking is a code-based feedback approach involving peer- and self-assessment, reducing the need to use alphanumeric grading while promoting the use of students’ metacognitive skills. The study involved a single cohort of 18,500 Key Stage 4 pupils (aged 14/15 at the start of the trial) and their English teachers (n = 990) in 103 secondary schools in England. The impact of the intervention was estimated as the difference in before and after measures of teacher workload, comparing teachers in 52 intervention schools and those in 51 control schools. The results suggest that the intervention had the effect of lessening teachers’ workload by reducing their working hours (effect size 0.16), including hours spent on marking and feedback (0.17). The intervention was largely implemented as designed and teachers were generally positive about the potential impact of FLASH on pupils’ learning outcomes.