Workload Metrics Research Articles

Evaluation of a 3D Solution Space-based ATC Workload Metric

Air Traffic Control (ATC) workload is a limiting factor for air traffic growth, creating a need for objective ATC workload metrics. Previous research has shown that the solution space diagram can be a basis for a workload prediction metric. The current solution space metric however, does not incorporate altitude. In this paper, a 3D solution space metric is described and evaluated. An experiment has been conducted to test the relation of the 3D solution space metric with workload and compare it to other workload metrics; the aircraft count, and a quasi-3D metric: the 2D layered solution space and the Instantaneous Self Assessment-based method. Weak correlations with workload were found for all tested metrics and no significant differences were found between them. Although no significant differences were found, the 2D layered metric showed better results than the 3D solution space-based metric, indicating that air traffic controllers might think in 2D layers over fixed altitude ranges rather than considering the complete 3D physical solution space.

Modelling the relationships between volume, intensity and injury-risk in professional rugby league players

ObjectiveThis study aimed to: (a) identify the association between external-workloads and injury-risk in the subsequent week; and (b) understand the effectiveness of workload variables in establishing injury-risk. DesignRetrospective cohort study. MethodsWorkload and injury data (soft-tissue) were collected from forty-eight professional male rugby league players. Load variables included duration (min), total distance (m), relative distance (mmin−1), high speed distance ([m]>20kmh−1), very-high speed distance ([m]>25kmh−1), acceleration and deceleration efforts (count) and PlayerLoad (Arbitrary Unit: AU). Cumulative two-, three- and four-weekly loads; Acute:Chronic Workload Ratio (ACWR); Mean-Standard Deviation Workload Ratio (MSWR) and strain values were calculated and divided into three equally-sized bins (low, moderate and high). Generalised Estimating Equations analysed relationships between workload variables and injury probability in the subsequent week. ResultsInjury-risk increased alongside increases in the ACWR for duration, total distance and PlayerLoad. Conversely, injury-risk decreased (Area Under Curve: 0.569–0.585) with increases in the four-weekly duration, total distance, accelerations, decelerations and PlayerLoad. For relative distance, high four-weekly workloads (high: >60mmin−1) demonstrated a positive association with injury-risk, whilst high two-weekly loads (high: >82 mmin−1) were negatively associated. ConclusionsA range of external workload metrics and summary statistics demonstrate either positive or negative associations with injury-risk status. Such findings provide the framework for the development of decision-support systems in which external workload metrics (e.g. total or high speed distance) can be uniquely and routinely monitored across a range of summary statistics (i.e. cumulative weekly loads and ACWR) in order to optimise player performance and welfare.

Measuring Workload in a Multitasking Environment Using Fractal Dimension of Pupil Dilation

ABSTRACTThe purpose of this study is to investigate the usefulness of the fractal dimension of pupil dilation as a measure of workload in a multitasking environment. Researchers have found that pupil dilation can reveal the underlying mechanism of the cognitive workload in a dynamic task environment. However, the workload metrics by using pupillary responses in a multitasking environment are not well studied in the literature. In this study, the participants’ pupil dilation was collected and analyzed by using a fractal analysis technique to assess the participants’ workload during the experiment. During the experiment, each participant performed a process monitoring task and Multi-Attribute Task Battery (MATB) task together. The process monitoring task simulated the responsibilities of process operators in the control room of oil and gas refinery plants. The MATB task consisted of system monitoring, target tracking, and dynamic resource management. To validate the outcome of this new metric, NASA-TLX questionnaires were used to measure the subjective workload, and the result of NASA-TLX was compared to the fractal dimension result. The findings from this study showed that the fractal dimension of pupil dilation could be used as a new physiological index to measure the workload in a multitasking environment.

Management of Medical Oncology Services in Canada: Redefined Workload With a Novel Supply-and-Demand Workforce Projection Model.

We developed a workforce-planning model to predict Canadian medical oncologist (MO) supply and clinical demand during the next 10 years. A forward calculation model was created to forecast the balance of MO supply and demand. MO supply was estimated by using Canadian Institute for Health Information, Canadian Medical Association, and Canadian Post-MD Education Registry data. Care demand was estimated by using data from Canadian Cancer Statistics and Alberta Cancer Registry. The Canadian Royal College MO Committee confirmed its face validity. The MO workforce is expected to grow from 541 staff in 2016 to 830 staff in 2026. During this period, new hires will increase from 39 to 56 per year, and departures will increase from 15 to 24 per year. Although cancer incidence rates will grow from 202,149 to 257,497, a projected increase in MO supply will mean fewer initial consultations, from an average of 168.5 consultations per MO in 2016 to 129.2 consultations per MO in 2026. The initiation of systemic therapy is projected to remain stable at 102.3 new systemic therapy starts per MO per year. We have developed a forward calculation MO workforce model that predicts a growing Canadian MO workforce and redefines MO workload dynamics. MO providers will increasingly support more follow-up care with the initiation of multiple lines of systemic therapy relative to the medical management of patients at the time of initial cancer diagnosis. Workload metrics, including follow-up and new therapy initiation rates, must be measured to appropriately to meet increasingly complex and growing care demands.

MIA: Metric Importance Analysis for Big Data Workload Characterization

Data analytics is at the foundation of both high-quality products and services in modern economies and societies. Big data workloads run on complex large-scale computing clusters, which implies significant challenges for deeply understanding and characterizing overall system performance. In general, performance is affected by many factors at multiple layers in the system stack, hence it is challenging to identify the key metrics when understanding big data workload performance. In this paper, we propose a novel workload characterization methodology using ensemble learning, called Metric Importance Analysis (MIA), to quantify the respective importance of workload metrics. By focusing on the most important metrics, MIA reduces the complexity of the analysis without losing information. Moreover, we develop the MIA-based Kiviat Plot (MKP) and Benchmark Similarity Matrix (BSM) which provide more insightful information than the traditional linkage clustering based dendrogram to visualize program behavior (dis)similarity. To demonstrate the applicability of MIA, we use it to characterize three big data benchmark suites: HiBench, CloudRank-D and SZTS. The results show that MIA is able to characterize complex big data workloads in a simple, intuitive manner, and reveal interesting insights. Moreover, through a case study, we demonstrate that tuning the configuration parameters related to the important metrics found by MIA results in higher performance improvements than through tuning the parameters related to the less important ones.

Towards Neuroadaptive Personal Learning Environments: Using fNIRS to Detect Changes in Attentional State

Towards Neuroadaptive Personal Learning Environments: Using fNIRS to Detect Changes in Attentional State

The level of mental workload related to the index of difficulty of the motor task and handedness

BACKGROUND: Manual movements are affected by the index of difficulty of the task and handedness. High index of difficulty and execution with the non-dominant hand can increase the mental workload.AIM: This study investigated the relation between the level of mental workload, the index of difficulty of the task, and handedness in manual movements.METHOD: Twenty-three right-handed male students (24.1 ± 4.3 years) participated in this study. The index of difficulty in the Grooved Pegboard task was manipulated by placing and removing the pegs and manipulation of handedness was made via right and left hand execution. Two mental workload metrics were obtained by electroencephalographic analysis: EEG-cognitive index, and EEG-workload index.RESULTS: The index of difficulty analyses indicated lower movement time when removing the pegs (low index of difficulty) compared to placing the pegs (high index of difficulty). EEG-workload index was higher when placing the pegs than removing. The handedness analyses indicated lower movement time with the right hand compared to the left hand. However, similar levels of mental workload were found.CONCLUSION: Results suggest higher mental effort when index of difficulty is increased, but not when the left non-dominant hand is performing the task.

Usability Comparison of Conventional Direct Control Versus Pattern Recognition Control of Transradial Prostheses

The goal of this study was to compare the usability of two control schemes for a transradial myoelectric prosthesis, including conventional direct control (DC) and pattern recognition (PR) control, when used by able-bodied individuals. Three types of response measures were captured to assess the control schemes, including learnability, performance, and cognitive workload. Prior research has applied performance and cognitive workload metrics for evaluation of prosthetics; however, workload measures applied in these studies (e.g., heart rate, electroencephalography, and respiration rate) have many limitations. This study used eye tracking to compare cognitive load implications of the different control schemes for a two degrees-of-freedom myoelectric prosthesis. In total, 12 participants were assigned to either control condition (six persons each) or perform a clothespin relocation task. Results revealed the PR scheme to be more intuitive for users and superior to DC across all response measures. We observed a lower learning percentage (i.e., greater learning potential), lower cognitive load, and greater productivity in task performance. This preliminary study illustrates efficacy of using eye-tracking-based measures of cognitive load and standardize test paradigms for assessment of upper limb prosthetic usability and supports PR prosthetic device control as an intuitive alternative to DC.

General Aviation Weather Alerting: The Effectiveness of Different Visual and Tactile Display Characteristics in Supporting Weather-Related Decision Making

ABSTRACTObjective: This research analyzed the effects of weather technology interface characteristics on the quality and timeliness of weather-related decision making via mental workload and situational awareness (SA) metrics using flight training devices (FTDs) and follow-up in-flight validation testing.Background: A significant number of fatal general aviation (GA) accidents are attributable to pilots flying into instrument meteorological conditions without the necessary certification for safe operation. Even when takeoff conditions are clear, they can degrade in flight. Pilots might consult in-flight weather technologies to inform flight-plan-related decisions.Method: GA pilots flew 2 scenarios in FTDs while receiving weather notifications (referred to here as alerts) via complex graphical or smart-watch-based visual displays and under different vibrotactile cuing conditions. In-flight testing examined if vibrotactile cuing effects found in simulation were present in real flight.Results: Results suggest the quality and timeliness of weather-related flight decisions and flight-related SA are better supported when displays use vibrotactile cues and embed alert text within maps rather than present it separately on a smart watch. In-flight data confirmed that pilots can reliably perceive vibrotactile cues in real aircraft.Conclusion: Vibrotactile cues improve the reception of notifications and can reduce cognitive demands while flying. Visual weather information should be displayed on larger screens that are positioned to reduce pilots’ postural reorientation. These findings can inform the design of weather technologies for supporting pilot in-flight decision making.

Real-Time Assessment of Operator State in Air Traffic Controllers Using Ocular Metrics

This study examined the utility of continuous operator state monitoring in predicting air traffic control officer (ATCO) workload and fatigue. Participants (N=8) were observed in live operational air traffic control environments for 60-minute periods. ATCO state was assessed using a real-time, computer vision-based system which tracked operator gaze and pupil diameter. Workload and fatigue were also assessed via the adapted Bedford Workload Scale and Samn-Perelli Fatigue Scale, respectively. Standard deviation of gaze was a significant predictor of both max and mean workload, showing a strong negative relationship with both subjective measures. Pupil diameter showed a significant positive relationship with operator fatigue. Our findings demonstrate the utility of continuous ocular metrics of workload and fatigue in operational environments.

Measuring the Impact of Console Operator Experience in a Simulated Petrochemical Refining Emergency Event

Critical process industries such as petrochemical refining have actively sought to make their operations safer and more efficient. In doing this, the industry has found success in automating systems. However, increasing levels of automation is known to have some negative effects on the human operator (Kaber & Endsley, 1997). Consequently, operators have had less opportunity to be exposed to and engage in managing emergency events due to reliable automation. The current investigation explores the role that console operator experience has in the management of emergency events and in the maintenance of situation awareness within the petrochemical industry. Incumbent Console Operators completed several trials of a simulated emergency event where prior exposure to the live event, performance, workload, and situation awareness metrics were collected. The results suggest that experience with managing the live event had little effect on the collected metrics. The potential implications of these results are discussed.

Subjective and Objective Measurement of Neonatal Nurse Practitioner Workload.

Neonatal nurse practitioner (NNP) workload is not well studied, and metrics specific to NNP practice are lacking. Factors such as changes in resident duty hours, increasing neonatal intensive care unit admissions, and a shortage of NNPs contribute to NNP workload. Increased workload has been shown to be detrimental to providers and can affect quality of care. This study quantified NNP workload using a subjective workload metric, the NASA Task Load Index, and a newly developed objective workload metric specific to NNP practice. The NNP group at a level IV academic medical center was studied. The sample included 22 NNPs and 47 workload experiences. A comparison of scores from the NASA Task Load Index and objective workload metric showed a moderate correlation (r = 0.503). Mental demand workload scores had the highest contribution to workload. Feelings of frustration also contributed to workload. The NASA Task Load Index can be utilized to measure the workload of NNPs. The objective workload metric has potential to quantify NNP workload pending further validation studies and is a simple, straightforward tool. Additional research is needed regarding NNP workload and methods to quantify workload. Larger studies are needed to validate the objective workload metric.

Simulation-Based Evaluation of the Effects of Patient Load on Mental Workload of Healthcare Staff.

In parts of Ohio, Veterans Affairs Medical Centers are working to handle patient load issues by sending patient overflows to the Wright-Patterson Medical Center. The Wright-Patterson Medical Center will benefit from the increase in patients; however, there are concerns that the patient quality of care may suffer. If the increase in patients results in the healthcare staff experiencing high mental workload levels, staff performance could be reduced. The objective of this research is to evaluate the influence of patient load on the mental workload of staff in an inpatient unit at the Wright-Patterson Medical Center. This research uses discrete-event simulation to quantitatively model the mental workload of healthcare staff in an inpatient unit of the Wright-Patterson Medical Center. The model was used to find the idle time, average workload, and overload time of healthcare staff under current and future patient loads. In addition, the performance of individual tasks was evaluated. The results of this research find a linear relationship between patient load and three workload metrics (idle time, average workload, and overload time) with each worsening as patient load increases. Nurses and technicians experience the greatest negative impacts to mental workload as patient load increases with those staff members who have the most workload at the baseline condition experiencing greater increase in workload as patient load increases. In addition, the time spent in an overload state increases disproportionately with patient load increases, with overload time increases being worse for urgent tasks than for nonurgent tasks. Based on this study, the researchers found that the modeled inpatient unit can safely handle the expected patient load increases. The study provides the unit with information to proactively prepare and reduce healthcare staff overloading.

OWL2 benchmarking for the evaluation of knowledge based systems.

OWL2 semantics are becoming increasingly popular for the real domain applications like Gene engineering and health MIS. The present work identifies the research gap that negligible attention has been paid to the performance evaluation of Knowledge Base Systems (KBS) using OWL2 semantics. To fulfil this identified research gap, an OWL2 benchmark for the evaluation of KBS is proposed. The proposed benchmark addresses the foundational blocks of an ontology benchmark i.e. data schema, workload and performance metrics. The proposed benchmark is tested on memory based, file based, relational database and graph based KBS for performance and scalability measures. The results show that the proposed benchmark is able to evaluate the behaviour of different state of the art KBS on OWL2 semantics. On the basis of the results, the end users (i.e. domain expert) would be able to select a suitable KBS appropriate for his domain.

Human-in-the-Loop Simulation Analysis of Integrated RPAS Operations in Trajectory Based Operations Environment

In this paper, Human-in-the-Loop (HiTL) simulations of Remotely Piloted Aircraft System (RPAS) operations in two different Air Traffic Management (ATM) concepts, conventional radar vectoring and Trajectory Based Operations (TBO), were performed to assess the impacts of RPAS integration in the future ATM environment. TBO concept maximizes the throughput by planning and sharing 4-D trajectories between pilots and controllers, and it is considered one of the key concepts to enable RPASs to operate with manned aircraft in congested airspaces. RPASs are characterized by having communication delay or temporary loss of communication. TBO capability was added to the integrated air traffic simulation system for this study, which was developed in the Inha University. HiTL simulations were performed by a trainee air traffic controller with three scenarios, and the data were analyzed using safety, efficiency, and controller workload metrics. The results suggest that TBO were effective in reducing delays and controller workload while maintaining the level of safety.

Measuring driving workload of heavy vehicles at roundabouts

Correctly designed roundabouts proved to have positive safety and functional performances. However, they are also affected by peculiar disadvantages. In particular, they are difficult to manoeuvre, especially for heavy vehicle drivers. Despite these concerns, there are currently no driving workload metrics devoted to roundabouts.A novel methodological approach is proposed for trying to quantify workload impinging on heavy vehicle drivers when manoeuvring through complex at-grade intersections. Proper acquisition of input data constitutes the starting point for future research about ascertainment of workload for these particular road scenarios. The described procedure enables recording steering wheel angles performed by a driver when manoeuvring an articulated lorry through a complex at-grade intersection. A field trial was carried out for verifying the practical feasibility of proposed method in capturing driver’s steering behaviour. Dynamic data acquired via global navigation satellite system instrumentation were related to actual driver’s steering wheel behaviour captured by camera frames. As a complement to the experiment, selected steering behaviour metrics were calculated. Steering Entropy attributed a high difficulty level to the manoeuvres performed through the roundabout, whereas High Frequency Component and Steering Reversal Rate showed intensity and occurrences of driver’s corrections needed for controlling position of the semitrailer at the ring. It appears that even a single roundabout may represent an arduous task for drivers. The study concludes with recommendations for further research about workload imposed by roundabouts to heavy vehicle drivers, with special attention to successions of closely spaced roundabouts.

Outcomes of a pharmacist-managed glucose collaborative practice agreement

The impact of a pharmacist-managed glucose collaborative practice agreement (CPA) on glycemic control at a tertiary medical center was investigated. A retrospective data analysis was performed on hospitalized, noncritically ill patients admitted between December 2012 and June 2014 who received at least one dose of subcutaneous insulin and experienced at least one blood glucose concentration of 140 mg/dL or higher. The study population was divided into cohorts based on admittance before versus after implementation of the CPA, as well as glucose management by pharmacist versus nonpharmacist provider. The primary endpoint of the study was glycemic control, defined as the percentage of total admitted days spent within a goal blood glucose range of 70-180 mg/dL. Secondary endpoints included the rate of hypoglycemia (less than 70 mg/dL), the rate of severe hypoglycemia (less than 40 mg/dL), the rate of severe hyperglycemia (greater than 300 mg/dL), the length of stay, and workload metrics. A total of 5146 patients were included in the study. There was no statistically significant difference in glycemic control across all cohorts (p > 0.05). Secondary outcomes showed no statistically significant differences in the rates of hypoglycemia, severe hypoglycemia, and severe hyperglycemia across all cohorts. There was a significantly longer length of stay in the pharmacist-managed cohort (p < 0.001). Workload metrics indicated a 25.8% increase in the number of pharmacist-managed glucose consults post-CPA implementation. Pharmacists at a tertiary medical center were able to provide an inpatient glucose management service that maintained similar glycemic control for patients with diabetes as nonpharmacist providers.

Workload metrics compliments for academic pursuits in pathology

Several clinical workload metrics have been developed and validated to provide a reasonable framework for compensation, workforce planning and accountability. Most Academic Pathologists have job descriptions that involve several diverse responsibilities outside the clinical service domain. Limited to the existing workload metrics, we continue to struggle not only amongst ourselves trying to structure our own complex and ever-growing work, but also communicating our needs with our academic system leaders. Moreover, the existing paradigm lacks the means for monitoring the safe boundaries or detecting “dangerous zones” at the pathologist's level as an interpretive operator beyond the mere quantification or qualification of their product (i.e. diagnosis). Occupational burnout for pathologists is not simply a function of their workload. This article highlights the need for complementary metrics for academic pursuits in pathology. It proposes a department-specific academic productivity complement and puts forth the idea of quantitating stress among pathologists. It is hoped that these two complements, used in concert with the existing clinical workload metrics, can inform decision making both at the individual and the system levels.

Comparison of three different techniques for camera and motion control of a teleoperated robot

This research aims to evaluate new methods for robot motion control and camera orientation control through the operator's head orientation in robot teleoperation tasks. Specifically, the use of head-tracking in a non-invasive way, without immersive virtual reality devices was combined and compared with classical control modes for robot movements and camera control. Three control conditions were tested: 1) a condition with classical joystick control of both the movements of the robot and the robot camera, 2) a condition where the robot movements were controlled by a joystick and the robot camera was controlled by the user head orientation, and 3) a condition where the movements of the robot were controlled by hand gestures and the robot camera was controlled by the user head orientation. Performance, workload metrics and their evolution as the participants gained experience with the system were evaluated in a series of experiments: for each participant, the metrics were recorded during four successive similar trials. Results shows that the concept of robot camera control by user head orientation has the potential of improving the intuitiveness of robot teleoperation interfaces, specifically for novice users. However, more development is needed to reach a margin of progression comparable to a classical joystick interface.

The temporal properties of e-learning: an exploratory study of academics ' conceptions

Purpose– The purpose of this paper is to present the results of an exploratory study that investigates Portuguese academics’ conceptions concerning the temporal properties of e-learning, in the context of traditional Higher Education Institutions.Design/methodology/approach– Grounded Theory methodology was used to systematically analyse data collected in semi-structured interviews with 62 academics. Data analysis followed the constant comparative method and its three-staged coding approach: open, axial and selective coding. Data collection and analysis developed until theoretical saturation was reached.Findings– Emergent academics’ conceptions concerning the temporal properties of e-learning indicate the existence of unregulated and unaccounted for dynamics, which are a direct consequence of transitioning from campus-based lecturing to teaching online using the affordances of virtual learning environments. This transition produces disruptions to established workload metrics and work patterns, as well as conflicts with dominant modes of instructional delivery that are not synchronised with the demands of online interaction and immediacy.Originality/value– This paper is valuable to e-learning strategists and administrators. An understanding of the temporal properties of e-learning as perceived by academics provides actionable knowledge that is directly applicable to the design of suitable work arrangements, normative frameworks and e-learning practices that address perceived time-related barriers. To mitigate the time-related conflicts and disruptions identified in the study the authors suggest the establishment of university-wide norms of virtual presence and the implementation of temporal protocols to sustain the productive engagement between instructors and learners.