Comparing methods for assessing operator functional state

Abstract

The assessment of an operator's functional state (i.e., the multidimensional pattern of human psychophysiological conditions that mediates performance) has great potential for increasing safety and reliability of critical systems. However, live monitoring of functional state using physiological and behavioral data still faces several challenges before achieving the level of precision required in many operational contexts. One open question is the level of granularity of the models. Is a general model sufficient or should subject-specific models be trained to ensure high accuracy? Another challenge concerns the formalization of a valid ground truth for training classifiers. This is critical in order to train models that are operationally relevant. This paper introduces the Decontextualized Dynamic Performance (DDP) metric which allows models to be trained simultaneously on different tasks using machine learning algorithms. This paper reports the performance of various classification algorithms at different levels of granularity. We compare a general model, task-specific models, and subject-specific models. Results show that the classification methods do not lead to statistically different performance, and that the predictive accuracy of subject-specific and task-specific models was actually comparable to a general model. We also compared various time-window sizes for the new DDP metric and found that results were degrading with a larger time window size.