Workload Estimation Research Articles

Computer-Aided Usability Evaluation of In-Vehicle Infotainment Systems

This paper reports the development of a computer-aided engineering (CAE) software for human machine interface (HMI) designers to predict and benchmark the usability of in-vehicle infotainment systems. At the front end of the software a graphic user interface (GUI) was developed that allows HMI designers to create digital mockups of designs and setup the tasks for simulation. At the back end a digital driver was created for simulating the driver cognition and performance based on the cognitive architecture of QN-MHP (Queuing Network-Model Human Processor). The software is able to simulate a driver performing in-vehicle secondary tasks (e.g., tuning radios) while steering a vehicle, make quantitative predictions of the driver’s task performance and workload. To validate the software outputs, an experiment was conducted on a driving simulator with two typical in-vehicle infotainment systems (a physical panel and a touch screen), and a radio tuning task was used as a test case. The results show that the software is able to generate task performance and workload estimates that are similar to the empirical data from human participants.

Detection of variations in cognitive workload using multi-modality physiological sensors and a large margin unbiased regression machine.

Physiological sensor based workload estimation technology provides a real-time means for assessing cognitive workload and has a broad range of applications in cognitive ergonomics, mental health monitoring, etc. In this paper we report a study on detecting changes in workload using multi-modality physiological sensors and a novel feature extraction and classification algorithm. We conducted a cognitive workload experiment involving multiple subjects and collected an extensive data set of EEG, ECG and GSR signals. We show that the GSR signal is consistent with the variations of cognitive workload in 75% of the samples. To explore cardiac patterns in ECG that are potentially correlated with the cognitive workload process, we computed various heart-rate-variability features. To extract neuronal activity patterns in EEG related to cognitive workload, we introduced a filter bank common spatial pattern filtering technique. As there can be large variations in e.g. individual responses to the cognitive workload, we propose a large margin unbiased recursive feature extraction and regression method. Our leave-one-subject-out cross validation test shows that, using the proposed method, EEG can provide significantly better prediction of the cognitive workload variation than ECG, with 87.5% vs 62.5% in accuracy rate.

EEG-based workload estimation across affective contexts.

Workload estimation from electroencephalographic signals (EEG) offers a highly sensitive tool to adapt the human–computer interaction to the user state. To create systems that reliably work in the complexity of the real world, a robustness against contextual changes (e.g., mood), has to be achieved. To study the resilience of state-of-the-art EEG-based workload classification against stress we devise a novel experimental protocol, in which we manipulated the affective context (stressful/non-stressful) while the participant solved a task with two workload levels. We recorded self-ratings, behavior, and physiology from 24 participants to validate the protocol. We test the capability of different, subject-specific workload classifiers using either frequency-domain, time-domain, or both feature varieties to generalize across contexts. We show that the classifiers are able to transfer between affective contexts, though performance suffers independent of the used feature domain. However, cross-context training is a simple and powerful remedy allowing the extraction of features in all studied feature varieties that are more resilient to task-unrelated variations in signal characteristics. Especially for frequency-domain features, across-context training is leading to a performance comparable to within-context training and testing. We discuss the significance of the result for neurophysiology-based workload detection in particular and for the construction of reliable passive brain–computer interfaces in general.

A method to estimate air traffic controller mental workload based on traffic clearances

Workload estimation is a complex domain which has been investigated extensively over the years. Past estimation techniques have focused on measuring workload directly from the air traffic controllers (ATCOs) or inferring it from traffic factors. The limitations of these techniques are interfering into the ATCO job and not being able to capture the differences amongst individual ATCOs respectively. This paper presents a novel technique overcoming these limitations, able to accurately estimate the workload experienced by the ATCO based exclusively on the clearances provided to air traffic. The technique, which was calibrated for the EUROCONTROL Maastricht Upper Area Control (MUAC) Centre, thereby has the potential to more accurately estimate actual airspace capacity. It is independent of the level of system automation and therefore applicable not only with the current ATM system, but also in the anticipated future highly automated environments as well as during the transition period. The paper discusses potential applications such as real time monitoring of operational workload and post-operations identification of sector workload imbalances. Both can contribute towards enhancing the performance of the ATM system.

Online learning of timeout policies for dynamic power management

Dynamic power management (DPM) refers to strategies which selectively change the operational states of a device during runtime to reduce the power consumption based on the past usage pattern, the current workload, and the given performance constraint. The power management problem becomes more challenging when the workload exhibits nonstationary behavior which may degrade the performance of any single or static DPM policy. This article presents a reinforcement learning (RL)-based DPM technique for optimal selection of timeout values in the different device states. Each timeout period determines how long the device will remain in a particular state before the transition decision is taken. The timeout selection is based on workload estimates derived from a Multilayer Artificial Neural Network (ML-ANN) and an objective function given by weighted performance and power parameters. Our DPM approach is further able to adapt the power-performance weights online to meet user-specified power and performance constraints, respectively. We have completely implemented our DPM algorithm on our embedded traffic surveillance platform and performed long-term experiments using real traffic data to demonstrate the effectiveness of the DPM. Our results show that the proposed learning algorithm not only adequately explores the power-performance trade-off with nonstationary workload but can also successfully perform online adjustment of the trade-off parameter in order to meet the user-specified constraint.

Toward a quantitative description of the neurodynamic organizations of teams

The goal was to develop quantitative models of the neurodynamic organizations of teams that could be used for comparing performance within and across teams and sessions. A symbolic modeling system was developed, where raw electroencephalography (EEG) signals from dyads were first transformed into second-by-second estimates of the cognitive Workload or Engagement of each person and transformed again into symbols representing the aggregated levels of the team. The resulting neurodynamic symbol streams had a persistent structure and contained segments of differential symbol expression. The quantitative Shannon entropy changes during these periods were related to speech, performance, and team responses to task changes.The dyads in an unscripted map navigation task (Human Communication Research Centre (HCRC) Map Task (MT)) developed fluctuating dynamics for Workload and Engagement, as they established their teamwork rhythms, and these were disrupted by external changes to the task. The entropy fluctuations during these disruptions differed in frequency, magnitude, and duration, and were associated with qualitative and quantitative changes in team organization and performance. These results indicate that neurodynamic models may be reliable, sensitive, and valid indicators of the changing neurodynamics of teams around which standardized quantitative models can begin to be developed.

Short-term cardiovascular measures for driver support: Increasing sensitivity for detecting changes in mental workload

With on-going increases in traffic density and the availability of more and more in-vehicle technology, driver overload is a growing concern. To reduce the burden of workload on the driver, it is essential that support systems that become available are able to use estimations of drivers' workload. In this paper a short-term cardiovascular approach to assess drivers' mental workload is described using data collected in a driving simulator study. The effects of short lasting increases in task demand (40s) on heart rate and blood pressure and derived variability measures are applied as indicators of mental effort. Fifteen drivers participated in 6 sessions of 1.5h in a driving simulator study. Two traffic density levels (7.5minute segments) were compared in which short-segments (40s) of fog were used to induce additional workload demands. Higher traffic density was reflected in increased systolic blood pressure and decreased blood pressure variability. Heart rate variability and blood pressure variability measures decreased during driving in fog in the low traffic condition, indicating increased effort investment during fog in this condition. The results show that the described short-term measures can be applied to give an indication of cardiovascular reactivity as a function workload.

Препроцессор языка Templet: инструмент программирования в терминах модели «процесс - сообщение»

Motivation: A large number of applications can be described as a set of processes that exchange messages. Traditionally the process-per-message model is used in the form of a specialized language or a run-time library for general purpose language. The first approach lacks implementation simplicity, while the second approach is difficult in use. We propose a new method that comprises domain-specific language called Templet for code markup, a general purpose language, and the preprocessor. Our approach is free from disadvantages mentioned above. Method: A code of a program is divided into blocks. Block boundaries are indicated by comments. The entire code structure is defined in Templet language so it can be checked out automatically before compilation. Description of Channels: A channel defines a message exchange protocol between two interconnected processes. We provide channel syntax in the form of Extended Backus-Naur Formalism (EBNF). The informational structure of the channel is described with Entity-Relation diagram (ER). Description of Processes: A process defines the algorithm for message processing. Information structure of the process is shown in conjunction with the syntax. EBNF and ER models are also used in the process specification. Syntax rules are illustrated with the fork-join code sample. Preprocessor structure and work scheme: We present the algorithm and the structure of the preprocessor. Subsystems discussed are: syntax analyzer; semantics analyzer; internal database; inference mechanism; and code generator. The method for estimation of workload of manual coding is presented. It shows the diminishment of workload in 20 times comparing with manual coding. Discussion: The preprocessor is used for skeleton programming as a part of web-service for automated parallel programming. Its advantages and features are discussed in comparison with parallelization with markup technique; general-purpose macro processor; parallel programming language, metaprogramming; and model-driven development. This paper is an extended version of a PIT 2014 paper [1].

Temporal fluctuations in driving demand: The effect of traffic complexity on subjective measures of workload and driving performance

Traffic density has been shown to be a factor of traffic complexity which influences driver workload. However, little research has systematically varied and examined how traffic density affects workload in dynamic traffic conditions. In this driving simulator study, the effects of two dynamically changing traffic complexity factors (Traffic Flow and Lane Change Presence) on workload were examined. These fluctuations in driving demand were then captured using a continuous subjective rating method and driving performance measures. The results indicate a linear upward trend in driver workload with increasing traffic flow, up to moderate traffic flow levels. The analysis also showed that driver workload increased when a lane change occurred in the drivers’ forward field of view, with further increases in workload when that lane change occurred in close proximity. Both of these main effects were captured via subjective assessment and with driving performance parameters such as speed variation, mean time headway and variation in lateral position. Understanding how these traffic behaviours dynamically influence driver workload is beneficial in estimating and managing driver workload. The present study suggests possible ways of defining the level of workload associated with surrounding traffic complexity, which could help contribute to the design of an adaptive workload estimator.

The Protocol Acuity Scoring Tool for Prediction of Emergency Medicine Research Study Workload

It is difficult to predict a priori how much work will be required for a proposed emergency medicine research study. This complicates resource allocation and feasibility assessment. The Protocol Acuity Scoring Tool (PAST) is a 32-item tool that assesses six major areas of conducting a clinical trial from startup to follow-up each on a scale from 1 to 3. It has been proposed as an instrument to predict the workload required for a proposed trial. We sought to determine the inter-rater reliability of the PAST score among multiple members of a research team and compare PAST scores to gestalt estimates of trial workload. Design: Prospective inter-rater reliability study. Raters: Four members of our clinical trials team each rated 5 different study protocols first by gestalt (using a range of 32-96) and then using the PAST score. The raters have a range of clinical trial experience (2 to 15 years) and training backgrounds (Clinical Research Coordinator, Registered Nurse, Medical Doctorate). The protocols included 3 randomized controlled treatment trials and 2 diagnostic blood marker studies that were familiar to the study team. Each protocol was independently scored by each rater using the PAST. Analysis: Free-marginal multirater kappa was calculated for inter-rater agreement for each PAST item. Gestalt estimates of protocol workload (mean, range and 95% confidence intervals) were compared to PAST scores. The range and median kappa for each major area are shown in Table 1. In general, the PAST score items showed poor inter-rater agreement with the exception of four items with a kappa >0.70: estimated enrollment, study arms, recruitment feasibility, and randomization. Six other items showed fair inter-rater agreement with a kappa >0.40. PAST scores tended to be lower and in a more narrow range than gestalt scores but were not significantly different. The PAST showed poor inter-rater agreement and did not provide different information from gestalt ratings. Future work should seek to tailor it to emergency medicine trials and create more reproducible items.Table 1Kappa ValuesNumber of ItemsMinimumMaximumMedianFeasibility10-0.050.750.23Pre-Start-up50.000.350.05Recruitment20.450.700.58Start-up40.250.700.45Study Process70.000.450.20Follow-up40.100.500.28 Open table in a new tab Table 2Gestalt and PAST Scores for each ProtocolStudies12345GestaltMean81.353.368.375.348.5Range73-8543-7058-8066-8540-6495% CI72.2 to 90.333.6 to 72.950.9 to 85.661.3 to 89.230.4 to 66.6PAST ScoresMean69.547.862.571.547.0Range55-8142-5160-6668-7646-4895% CI51.9 to 87.141.5 to 54.057.7 to 67.365.6 to 77.445.7 to 48.395% CI= 95% Confidence Intervals Open table in a new tab

Estimating cognitive workload using wavelet entropy-based features during an arithmetic task

Electroencephalography (EEG) has shown promise as an indicator of cognitive workload; however, precise workload estimation is an ongoing research challenge. In this investigation, seven levels of workload were induced using an arithmetic task, and the entropy of wavelet coefficients extracted from EEG signals is shown to distinguish all seven levels. For a subject-independent multi-channel classification scheme, the entropy features achieved high accuracy, up to 98% for channels from the frontal lobes, in the delta frequency band. This suggests that a smaller number of EEG channels in only one frequency band can be deployed for an effective EEG-based workload classification system. Together with analysis based on phase locking between channels, these results consistently suggest increased synchronization of neural responses for higher load levels.

Prediction of Perceived Workload From Task Performance and Heart Rate Measures

To improve operator efficiency and effectiveness, designers increasingly apply automation to allocate tasks once performed by human operators to the system. Unfortunately, these systems are often complex, potentially imposing increased mental task load on the operator, or placing the operator in a supervisory role where they can become overly dependent on automation. A proposed solution is adaptive automation, which increases automation when an operator is overloaded, and disabled as the operator has spare mental capacity. Changes in performance and physiological measures have shown promise in triggering changes in automation levels. However, the literature lacks well-documented or consistently supported measures for mental workload prediction. The present work sought to define a model which could predict perceived workload as a function of performance and heart rate measures by imposing various levels of task loading on a group of individuals while monitoring their performance, recording their heart rate information with an electrocardiogram and obtaining subjective estimates of mental workload. Heart rate (HR) and several heart rate variability (HRV) measurements where significantly affected by Task Load. This paper describes a linear regression model for predicting participants’ perceived workload as a function of a proposed summary performance metric and HR measures.

Towards the Development of Quantitative Descriptions of the Neurodynamic Rhythms and Organizations of Teams

The goal was to begin developing quantitative approaches for modeling the neurodynamic rhythms and organizations of teams. Raw EEG signals from team members were first transformed into es-timates of cognitive workload and transformed again into neurodynamics symbols showing the second-by-second workload of each individual as well as the team. Periods of increased or decreased symbol organiza-tion in the data streams were hypothesized to reflect periods of increased or decreased organization around the cognitive construct of workload. These segments were identified by a moving average smoothing of the Shannon entropy over the length of the performance and then related to team speech, actions and team responses to endogeneous and exogeneous task changes. Two-person teams in an unscripted map naviga-tion task developed a common, dominant coordination dynamic for workload whose rhythm was disrupted by exogeneous changes to the task. The entropy fluctuations during these disruptions differed in magnitude and duration within and across performances and were associated with qualitative and quantitative changes in team organization. Similar results were obtained with three and six person teams on other complex tasks. These results indicate that neurodynamic measures may be reliable, sensitive and valid indicators of the changing neurodynamics of teams around which standardized quantitative models can be developed.

Workload estimation of harvesters during the operations of work cycle

AbstractThe research is devoted to the investigation of working efficiency and workload of harvesters. The complex criterion of their efficiency evaluation is suggested. The estimation of operational effectiveness of the harvester MLH-414 (MЛX-414) in various working conditions has been made. The various styles of limbing have been analysed. The recommendations for the efficient application of a work mix have been given. Mathematical modelling of harvester loading during transport and engineering operations has been carried out. The adequacy of the designed model of the actual harvester structure has been verified. The most overloaded operations of the harvester's working cycle are defined. The ways for decreasing the emerging loadings are offered.

Integrated wall stress: a new methodological approach to assess ventricular workload and myocardial contractile reserve

BackgroundWall stress is a useful concept to understand the progression of ventricular remodeling. We measured cumulative LV wall stress throughout the cardiac cycle over unit time and tested whether this “integrated wall stress (IWS)” would provide a reliable marker of total ventricular workload.Methods and resultsWe applied IWS to mice after experimental myocardial infarction (MI) and sham-operated mice, both at rest and under dobutamine stimulation. Small infarcts were created so as not to cause subsequent overt hemodynamic decompensation. IWS was calculated over one minute through simultaneous measurement of LV internal diameter and wall thickness by echocardiography and LV pressure by LV catheterization. At rest, the MI group showed concentric LV hypertrophy pattern with preserved LV cavity size, LV systolic function, and IWS comparable with the sham group. Dobutamine stimulation induced a dose-dependent increase in IWS in MI mice, but not in sham mice; MI mice mainly increased heart rate, whereas sham mice increased LV systolic and diastolic function. IWS showed good correlation with a product of peak-systolic wall stress and heart rate. We postulate that this increase in IWS in post-MI mice represents limited myocardial contractile reserve.ConclusionWe hereby propose that IWS provides a useful estimate of total ventricular workload in the mouse model and that increased IWS indicates limited LV myocardial contractile reserve.

Towards database virtualization for database as a service

Advances in operating system and storage-level virtualization technologies have enabled the effective consolidation of heterogeneous applications in a shared cloud infrastructure. Novel research challenges arising from this new shared environment include load balancing, workload estimation, resource isolation, machine replication, live migration, and an emergent need of automation to handle large scale operations with minimal manual intervention. Given that databases are at the core of most applications that are deployed in the cloud, database management systems (DBMSs) represent a very important technology component that needs to be virtualized in order to realize the benefits of virtualization from autonomic management of data-intensive applications in large scale data-centers. The goal of this tutorial is to survey the techniques used in providing elasticity in virtual machine systems, shared storage systems, and survey database research on multitenant architectures and elasticity primitives. This foundation of core Database as a Service advances, together with a primer of important related topics in OS and storage-level virtualization, are central for anyone that wants to operate in this area of research.

A Fuzzy-Based Dynamic Load Decision Making Scheme in Cloud Computing

In this paper, a fuzzy-based dynamic resource allocation scheme for load balancing in cloud computing (FDLB) is presented to growth of the service type and user number in the mobile networks of the higher performance is required in service provision and throughput. Besides, the estimation of workload is difficult and time consuming, and the result causes an unstable system and unnecessary message passing overhead. The structure of the FDLB is composed of the design the load distributed agent (LDA). It is observed that the FDLB exhibits better adaptability, robustness, and fault-tolerant capability, thus yielding better performances compared to other existing algorithms.

Managing Overloaded Hosts for Dynamic Consolidation of Virtual Machines in Cloud Data Centers under Quality of Service Constraints

Dynamic consolidation of virtual machines (VMs) is an effective way to improve the utilization of resources and energy efficiency in cloud data centers. Determining when it is best to reallocate VMs from an overloaded host is an aspect of dynamic VM consolidation that directly influences the resource utilization and quality of service (QoS) delivered by the system. The influence on the QoS is explained by the fact that server overloads cause resource shortages and performance degradation of applications. Current solutions to the problem of host overload detection are generally heuristic based, or rely on statistical analysis of historical data. The limitations of these approaches are that they lead to suboptimal results and do not allow explicit specification of a QoS goal. We propose a novel approach that for any known stationary workload and a given state configuration optimally solves the problem of host overload detection by maximizing the mean intermigration time under the specified QoS goal based on a Markov chain model. We heuristically adapt the algorithm to handle unknown nonstationary workloads using the Multisize Sliding Window workload estimation technique. Through simulations with workload traces from more than a thousand PlanetLab VMs, we show that our approach outperforms the best benchmark algorithm and provides approximately 88 percent of the performance of the optimal offline algorithm.

OCENA OBCIĄŻENIA UKŁADU RUCHU U ŁADOWACZY NIECZYSTOŚCI STAŁYCH

The aim of this work was to assess the load on the musculoskeletal system and its effects in the collectors of solid refuse. The rationale behind this study was to formulate proposals how to reduce excessive musculoskeletal load in this group of workers. The study group comprised 15 refuse collectors aged 25 to 50 years. Data about the workplace characteristics and subjective complaints of workers were collected by the free interview and questionnaire. During the survey the photorecording of the workpostures, the distance and velocity by GPS recorders, measurements of forces necessary to move containers, energy expenditure (lung ventilation method), workload estimation using the Firstbeat system and REBA method and stadiometry were done. The distance walked daily by the collectors operating in terms of 2 to 3 in urban areas was about 15 km, and in rural areas about 18 km. The most frequent musculoskeletal complaints concerned the feet (60% subjects), knees, wrists and shoulders (over 40% subjects). After work-shift all examined workers had vertebral column shorter by 10 to 14 mm (11.4 mm mean). The results of our study show that the refuse collectors are subjected to a very high physical load because of the work organization and the way it is performed. To avoid adverse health effects and overload it is necessary to undertake ergonomic interventions, involving training of workers to improve the way of their job performance, active and passive leisure, technical control of the equipment and refuse containers, as well as the renegotiation of contracts with clients, especially those concerning non-standard containers.

Fuzzy-TLX: using fuzzy integrals for evaluating human mental workload with NASA-Task Load indeX in laboratory and field studies

The aim of this study was to assess mental workload in which various load sources must be integrated to derive reliable workload estimates. We report a new algorithm for computing weights from qualitative fuzzy integrals and apply it to the National Aeronautics and Space Administration -Task Load indeX (NASA-TLX) subscales in order to replace the standard pair-wise weighting technique (PWT). In this paper, two empirical studies were reported: (1) In a laboratory experiment, age- and task-related variables were investigated in 53 male volunteers and (2) In a field study, task- and job-related variables were studied on aircrews during 48 commercial flights. The results found in this study were as follows: (i) in the experimental setting, fuzzy estimates were highly correlated with classical (using PWT) estimates; (ii) in real work conditions, replacing PWT by automated fuzzy treatments simplified the NASA-TLX completion; (iii) the algorithm for computing fuzzy estimates provides a new classification procedure sensitive to various variables of work environments and (iv) subjective and objective measures can be used for the fuzzy aggregation of NASA-TLX subscales. Practitioner Summary: NASA-TLX, a classical tool for mental workload assessment, is based on a weighted sum of ratings from six subscales. A new algorithm, which impacts on input data collection and computes weights and indexes from qualitative fuzzy integrals, is evaluated through laboratory and field studies. Pros and cons are discussed.