AbstractDetecting the human operator's cognitive state is paramount in settings wherein maintaining optimal workload is necessary for task performance. Blink rate is an established metric of cognitive load, with a higher blink frequency being observed under conditions of greater workload. Measuring blink rate requires the use of eye‐trackers which limits the adoption of this metric in the real‐world. The authors aim to investigate the effectiveness of using a generic camera‐based system as a way to assess the user's cognitive load during a computer task. Participants completed a mental task while sitting in front of a computer. Blink rate was recorded via both the generic camera‐based system and a scientific‐grade eye‐tracker for validation purposes. Cognitive load was also assessed through the performance in a single stimulus detection task. The blink rate recorded via the generic camera‐based approach did not differ from the one obtained through the eye‐tracker. No meaningful changes in blink rate were however observed with increasing cognitive load. Results show the generic‐camera based system may represent a more affordable, ubiquitous means for assessing cognitive workload during computer task. Future work should further investigate ways to increase its accuracy during the completion of more realistic tasks.

A general testing and analysis framework for the Step-Stress fatigue testing method is identified, utilizing interval-censored data and maximum likelihood estimation in an effort to improve estimation of fatigue strength distribution parameters has been performed. The Step-Stress method’s limitations are characterized, using a simple material model that considers cumulative damage to evaluate load history effects. In this way, the performance including cumulative damage was evaluated and quantified using a probabilistic approach with Monte-Carlo simulations, benchmarked against the Staircase method throughout the work. It was found that the Step-Stress method, even when cumulative damage occurs to a wide extent, outperforms the Staircase method, especially for small sample sizes. Furthermore, positive results reaches further than the increase performance in estimating fatigue strength distribution parameters, where improvements in secondary information, i.e. S-N data gained from failure specimens, are shown to be distributed more closely to the fatigue life region of interest.

Modern smart assembly lines commonly include electric tools with built-in sensors to tighten safety-critical joints. These sensors generate data that are subsequently analyzed by human experts to diagnose potential tightening errors. Previous research aimed to automate the diagnosing process by developing diagnosing models based on tightening theory and calibration of the friction coefficient in specific lab setups. Generalizing these results is difficult and often unsuccessful since friction coefficients vary between lab and production environments. To overcome this problem, this paper presents a novel methodology that builds multi-label classification deep learning models for diagnosing tightening errors using production data. The proposed methodology comprises three key contributions, i.e., the Labrador method, the Model Combo (MoBo) framework, and a heuristic evaluation method. Labrador is an elastic deep learning based sensor fusion method that (1) uses feature encoders to extract features; (2) conducts data-level and/or feature-level sensor fusion in both time and frequency domains; and (3) performs multi-label classification to detect and diagnose tightening errors. MoBo is a configurable and modular framework that supports Labrador in identifying optimal feature encoders. With MoBo and Labrador, one can easily explore and design a bounded search space for sensor fusion strategies (SFSs) and feature encoders. In order to identify the optimal solution within the defined search space, this paper introduces a heuristic method. By evaluating the trade-off between machine learning (ML) metrics (e.g., accuracy, subset accuracy, and F1) and operational (OP) metrics (e.g., inference latency), the proposed method identifies the most suitable solution depending on the requirements of individual use cases. In the experimental evaluation, we adopt the proposed methodology to identify the most suitable multi-label classification solutions for diagnosing tightening errors. To optimize ML metrics, the identified solution achieved 99.69% accuracy, 93.39% subset accuracy, 97.39% F1, and 6.68ms inference latency. To optimize OP metrics, the identified solution achieved 99.66% accuracy, 92.65% subset accuracy, 97.28% F1, and 2.41ms inference latency.

This work aims to validate using a video-based blink detection system for detecting changes in cognitive load. Participants completed a cognitive task with increasing levels of difficulty. Blink rate was recorded via both our video-based system and a scientific-grade eye-tracker. Results showed no differences in the blink rates recorded by the two systems. However, while strong evidence was found that the blink rate recorded through the eye-tracker increased under greater cognitive task demand, no such pattern was observed for the output from the video-based system. Our findings show that the overall performance of the video-based system was comparable with that of the eye-tracker's, but its sensitivity in detecting changes in cognitive load was inferior to that of its counterpart's.

This work investigates using pupil size to track changes in physical fatigue during manufacturing tasks. Participants completed an automotive manufacturing task over three trials. Impulse and peak force were recorded as ground-truth metrics of physical fatigue. Pupil size and blink rate were also recorded by means of a head-mounted eye-tracker. Impulse and peak force decreased between trial 1 and trial 3 suggesting an increase in fatigue. Interestingly, this was accompanied by a significant reduction in pupil size. No difference in blink rate was found. Our study adds to the literature on fatigue assessment suggesting pupil size as a future possible alternative metric for fatigue assessment in manufacturing Ergonomics.

The life and condition of a mine truck frame are related to how the machine is used. Damage from stress cycles is accumulated over time, and measurements throughout the life of the machine are needed to monitor the condition. This results in high demands on the durability of sensors, especially in a harsh mining application. To make a monitoring system cheap and robust, sensors already available on the vehicles are preferred rather than additional strain gauges. The main question in this work is whether the existing on-board sensors can give the required information to estimate stress signals and calculate accumulated damage of the frame. Model complexity requirements and sensors selection are also considered. A final question is whether the accumulated damage can be used for prognostics and to increase reliability. The investigation is performed using a large data set from two vehicles operating in real mine applications. Coherence analysis, ARX-models, and rain flow counting are techniques used. The results show that a low number of available on-board sensors like load cells, damper cylinder positions, and angle transducers can give enough information to recreate some of the stress signals measured. The models are also used to show significant differences in usage by different operators, and its effect on the accumulated damage.

Force output and muscle activity represent the gold standards for measuring physical fatigue. This study explores using ocular metrics for tracking changes in physical fatigue during the completion of a repeated handle push/pull task. Participants completed this task over three trials, and pupil size was recorded by means of a head-mounted eye-tracker. Blink frequency was also measured. Force impulse and maximum peak force were used as ground-truth measures of physical fatigue. As expected, a reduction in peak force and impulse was observed over time as participants became more fatigued. More interestingly, pupil size was also found to decrease from trial 1 through trial 3. No changes in blink rate were found with increasing physical fatigue. While exploratory in nature, these findings add to the sparse literature exploring the use of ocular metrics in Ergonomics. They also advance the use of pupil size as a possible future alternative for physical fatigue detection.