What methods can be used to quantify workload?

Pathologist workload in the United States has traditionally been measured by relative value units (RVUs), which is often criticized for providing an inaccurate estimate of actual work. This study compares three methods for measuring workload. Surgical pathology and cytopathology workload for 1 representative month at Stanford Health Care was assessed using three different methods: RVUs, Royal College of Pathologists (RCP) point system, and University of Washington-Seattle (UW) slide count method. Pearson linear regression analysis showed a strong positive correlation of RVUs with the RCP (0.93, P < .01) and UW (0.86, P < .01) systems. The correlation between the RCP and UW systems was weaker (0.70, P = .05). The RCP system rated gastrointestinal, genitourinary, and breast workload lower than the RVU system while medical liver/renal and cytology were valued higher. The UW system overvalued breast workload. RCP is the most advanced and well-developed system for evaluating workload. It provides more weight for higher complexity specimens, while RVUs favor specialties with higher volume of small specimens, and slide counts favor specialties with extensively sampled large specimens.

Today's work environments have high cognitive demands, and mental workload is one of the main causes of work stress, human errors, and accidents. While several mental workload studies have compared the mental workload perceived by groups of experienced participants to that perceived by novice groups, no comparisons have been made between the same individuals performing the same tasks at different times. This work aims to compare NASA Task Load Index (NASA-TLX) to Workload Profile (WP) in terms of their sensitivity. The comparison considers the impact of experience and task differentiation in the same individual once a degree of experience has been developed in the execution of the same tasks. It also considers the acceptability and intrusivity of the techniques. The sample consisted of 30 participants who performed four tasks in two sessions. The first session was performed when participants had no experience; the second session was performed after a time of practice. Mental workload was assessed after each session. Statistical methods were used to compare the results. The NASA-TLX proved to be more sensitive to experience, while the WP showed greater sensitivity to task differentiation. In addition, while both techniques featured a similar degree of intrusivity, the NASA-TLX received greater acceptability. The acceptability of WP is low due to the high complexity of its dimensions and clarifying explanations of these may be necessary to increase acceptability. Future research proposals should be expanded to consider mental workload when designing work environments in current manufacturing environments.

Personal and ubiquitous healthcare applications offer new opportunities to prevent long-term health damage due to increased mental workload by continuously monitoring physiological signs related to prolonged high workload and providing just-in-time feedback. In order to achieve a quantification of mental load, different load levels that occur during a workday have to be discriminated. In this work, we present how mental workload levels in everyday life scenarios can be discriminated with data from a mobile ECG logger by incorporating individual calibration measures. We present an experiment design to induce three different levels of mental workload in calibration sessions and to monitor mental workload levels in everyday life scenarios of seven healthy male subjects. Besides the recording of ECG data, we collect subjective ratings of the perceived workload with the NASA Task Load Index (TLX), whereas objective measures are assessed by collecting salivary cortisol. According to the subjective ratings, we show that all participants perceived the induced load levels as intended from the experiment design. The heart rate variability (HRV) features under investigation can be classified into two distinct groups. Features in the first group, representing markers associated with parasympathetic nervous system activity, show a decrease in their values with increased workload. Features in the second group, representing markers associated with sympathetic nervous system activity or predominance, show an increase in their values with increased workload. We employ multiple regression analysis to model the relationship between relevant HRV features and the subjective ratings of NASA-TLX in order to predict the mental workload levels during office-work. The resulting predictions were correct for six out of the seven subjects. In addition, we compare the performance of three classification methods to identify the mental workload level during office-work. The best results were obtained with linear discriminant analysis (LDA) that yielded a correct classification for six out of the seven subjects. The k-nearest neighbor algorithm (k-NN) and the support vector machine (SVM) resulted in a correct classification of the mental workload level during office-work for five out of the seven subjects.

Neonatal nurse practitioner (NNP) workload is not well studied, and metrics specific to NNP practice are lacking. Factors such as changes in resident duty hours, increasing neonatal intensive care unit admissions, and a shortage of NNPs contribute to NNP workload. Increased workload has been shown to be detrimental to providers and can affect quality of care. This study quantified NNP workload using a subjective workload metric, the NASA Task Load Index, and a newly developed objective workload metric specific to NNP practice. The NNP group at a level IV academic medical center was studied. The sample included 22 NNPs and 47 workload experiences. A comparison of scores from the NASA Task Load Index and objective workload metric showed a moderate correlation (r = 0.503). Mental demand workload scores had the highest contribution to workload. Feelings of frustration also contributed to workload. The NASA Task Load Index can be utilized to measure the workload of NNPs. The objective workload metric has potential to quantify NNP workload pending further validation studies and is a simple, straightforward tool. Additional research is needed regarding NNP workload and methods to quantify workload. Larger studies are needed to validate the objective workload metric.

PurposeTo quantify, and compare, workload for several common physician-based treatment planning tasks using objective and subjective measures of workload. To assess the relationship between workload and performance to define workload levels where performance could be expected to decline. Methods and MaterialsNine physicians performed the same 3 tasks on each of 2 cases (“easy” vs “hard”). Workload was assessed objectively throughout the tasks (via monitoring of pupil size and blink rate), and subjectively at the end of each case (via National Aeronautics and Space Administration Task Load Index; NASA-TLX). NASA-TLX assesses the 6 dimensions (mental, physical, and temporal demands, frustration, effort, and performance); scores > or ≈ 50 are associated with reduced performance in other industries. Performance was measured using participants’ stated willingness to approve the treatment plan. Differences in subjective and objective workload between cases, tasks, and experience were assessed using analysis of variance (ANOVA). The correlation between subjective and objective workload measures were assessed via the Pearson correlation test. The relationships between workload and performance measures were assessed using the t test. ResultsEighteen case-wise and 54 task-wise assessments were obtained. Subjective NASA-TLX scores (P < .001), but not time-weighted averages of objective scores (P > .1), were significantly lower for the easy vs hard case. Most correlations between the subjective and objective measures were not significant, except between average blink rate and NASA-TLX scores (r = −0.34, P = .02), for task-wise assessments. Performance appeared to decline at NASA-TLX scores of ≥55. ConclusionsThe NASA-TLX may provide a reasonable method to quantify subjective workload for broad activities, and objective physiologic eye-based measures may be useful to monitor workload for more granular tasks within activities. The subjective and objective measures, as herein quantified, do not necessarily track each other, and more work is needed to assess their utilities. From a series of controlled experiments, we found that performance appears to decline at subjective workload levels ≥55 (as measured via NASA-TLX), which is consistent with findings from other industries.

When using a upper-limb prosthesis, mental, emotional, and physical effort is often experienced. These have been linked to high rates of device dissatisfaction and rejection. Therefore, understanding and quantifying the complex nature of workload experienced when using, or learning to use, a upper-limb prosthesis has practical and clinical importance for researchers and applied professionals. The aim of this paper was to design and validate a self-report measure of mental workload specific to prosthesis use (The Prosthesis Task Load Index; PROS-TLX) that encapsulates the array of mental, physical, and emotional demands often experienced by users of these devices. We first surveyed upper-limb prosthetic limb users who confirmed the importance of eight workload constructs taken from published literature and previous workload measures. These constructs were mental demands, physical demands, visual demands, conscious processing, frustration, situational stress, time pressure and device uncertainty. To validate the importance of these constructs during initial prosthesis learning, we then asked able-bodied participants to complete a coin-placement task using their anatomical hand and then using a myoelectric prosthesis simulator under low and high mental workload. As expected, using a prosthetic hand resulted in slower movements, more errors, and a greater tendency to visually fixate the hand (indexed using eye-tracking equipment). These changes in performance were accompanied by significant increases in PROS-TLX workload subscales. The scale was also found to have good convergent and divergent validity. Further work is required to validate whether the PROS-TLX can provide meaningful clinical insights to the workload experienced by clinical users of prosthetic devices.

Mental workload is a major consideration for the design of emergency operation procedures (EOPs) in nuclear power plants. Continuous and objective measures are desired. This paper compares seven mental workload measurement methods (pupil size, blink rate, blink duration, heart rate variability, parasympathetic/sympathetic ratio, total power and (Goals, Operations, Methods, and Section Rules)-(Keystroke Level Model) GOMS-KLM-based workload index) with regard to sensitivity, validity and intrusiveness. Eighteen participants performed two computerised EOPs of different complexity levels, and mental workload measures were collected during the experiment. The results show that the blink rate is sensitive to both the difference in the overall task complexity and changes in peak complexity within EOPs, that the error rate is sensitive to the level of arousal and correlate to the step error rate and that blink duration increases over the task period in both low and high complexity EOPs. Cardiac measures were able to distinguish tasks with different overall complexity. The intrusiveness of the physiological instruments is acceptable. Finally, the six physiological measures were integrated using group method of data handling to predict perceived overall mental workload. Practitioner Summary: The study compared seven measures for evaluating the mental workload with emergency operation procedure in nuclear power plants. An experiment with simulated procedures was carried out, and the results show that eye response measures are useful for assessing temporal changes of workload whereas cardiac measures are useful for evaluating the overall workload.

We aimed to provide an assessment of the impact of workload manipulations on various cardiac measurements. We further sought to determine the most effective measurement approaches of cognitive workload as well as quantify the conditions under which these measures are most effective for interpretation. Cognitive workload affects human performance, particularly when load is relatively high (overload) or low (underload). Despite ongoing interest in assessing cognitive workload through cardiac measures, it is currently unclear which cardiac-based assessments best indicate cognitive workload. Although several quantitative studies and qualitative reviews have sought to provide guidance, no meta-analytic integration of cardiac assessment(s) of cognitive workload exists to date. We used Morris and DeShon's meta-analytic procedures to quantify the changes in cardiac measures due to task load conditions. Sample-weighted Cohen's d values suggest that several metrics of cardiac activity demonstrate sensitivity in response to cognitive workload manipulations. Heart rate variability measures show sensitivity to task load, conditions of event rate, and task duration. Authors of future work should seek to quantify the utility of leveraging multiple metrics to understand workload. Results suggest that assessment of cognitive workload can be done using various cardiac activity indicators. Further, given the number of valid and reliable measures available, researchers and practitioners should base their selection of a psychophysiological measure on the experimental and practical concerns inherent to their task/protocol. Findings bear implications for future assessment of cognitive workload within basic and applied settings. Future research should seek to validate conditions under which measurements are best interpreted, including but not limited to individual differences.

Minimal research has been conducted to develop non-invasive processes for quantifying and evaluating worker mental workload - a critical concern - at the task level in the construction industry. One reason for this gap in research is the complex and dynamic nature of the construction process, which makes construction work more complicated to measure and predict compared to work in other industries. This paper presents a novel approach and corresponding conceptual model to quantify and evaluate construction worker perception of mental workload at the task level using the energy concept. A conceptual process for assessing mental workload (MWL), i.e., the feeling of stress, pressure, and being overwhelmed due to the task nature, factors, conditions, and resources that accompany the performance of the task, was developed from extant research and interviews. The Delphi method was utilized to characterize the energy-based model and provide initial verification. The results from the literature review, expert insight, and four rounds of the Delphi survey revealed 14 constituents, 51 components, and one metric for each component to measure the level of MWL felt by a worker. These constituents, components, and metrics were used to develop a model for measuring construction worker MWL. This study contributes to knowledge by developing a novel non-invasive method for assessing potential task-level MWL using an energy-based model. The energy-based assessment model contributes to practice by providing a tool that could be used to measure the potential impact of construction tasks on workers perceived mental workload.

245 Background: Workload is a hypothetical construct representing the overall human cost incurred during a task. Across disciplines, the NASA Task Load Index (TLX) has been used as a subjective measure of workload. It quantifies six dimensions of work (mental, physical, and temporal demands, frustration, effort, and performance) on a scale of 1-100 scale, with scores &gt;55 (i.e. overworked) associated with lesser performance. We herein quantify differences in physician workload for planning a relatively “simple” (palliative 2 field opposed lateral brain) versus relatively “complex” (curative 4 field pancreas) case. Further, we explore the association between workload and performance to define TLX levels where performance could be expected to decline. Methods: Nine physicians planned two cases, each case involving three tasks (i.e., task 1 - review written patient medical records; task 2 - review diagnostic images and design of treatment fields; and task 3 - review and approval of the treatment plan). After completing each case, TLX scores of each case/task was collected. Differences in workload between cases/tasks were assessed via ANOVA. The association between workload and performance (assessed using time-to-task completion and self-assessments) was assessed via Pearson’s correlation test. Results: 54 TLX workload and performance assessments were obtained. Workloads for the simple brain case/tasks averaged 41-48, versus 62-69 for the complex pancreas case/tasks (p&lt;0.001 via ANOVA). There were no differences between tasks for the individual cases (p&gt;0.1). There was a correlation between TLX scores and time-to-complete (r=0.54, p&lt;0.001), and with physician self-assessments (r=-0.74, p&lt;0.001). There was a trend towards reduced performance with TLX scores &gt; 55. Conclusions: Physician workload levels are markedly lower for “simple” versus “complex” cases, indicating that TLX is a reliable tool to quantify workload. Performance appears to decline at TLX levels &gt; 55, which is consistent with findings from other industries. Thus, we propose workload assessment (via TLX) to be considered as an independent quality measure to assess the quality assurance (QA) of processes used to deliver radiation therapy.