Human Task Performance Research Articles

Auditory Noise Facilitates Lower Visual Reaction Times in Humans.

Noise is commonly seen as a disturbance but can influence any system it interacts with. This influence may not always be desirable, but sometimes it can improve the system's performance. For example, stochastic resonance is a phenomenon where adding the right amount of noise to a weak signal makes it easier to detect. This is known as sub-threshold detection. This sub-threshold detection's natural fingerprint is the fact that the threshold values follow an inverse U-shaped curve as the noise intensity increases. The minimum threshold value is the point of maximum sensitivity and represents the optimal point that divides the dynamics in two. Below that point, we can find the beneficial noise branch, where the noise can facilitate better detection. Above that point, the common detrimental noise concept can be found: adding noise hinders signal detection. The nervous system controls the movements and bodily functions in the human body. By reducing the sensory thresholds, we can improve the balance of these functions. Additionally, researchers have wondered if noise could be applied to different senses or motor mechanisms to enhance our abilities. In this work, noise is used to improve human reaction times. We tested the hypothesis that visual reaction times decrease significantly when the subject's perception is in the beneficial noise branch and closer to the optimal point than outside of this condition. Auditory noise was introduced in 101 human subjects using an interface capable of searching for the right amount of noise to place the subject in the beneficial noise branch close to the optimal point. When comparing the results, the reaction times decreased when the subjects were at the optimal point compared to when the subjects were outside of such conditions. These results reveal the possibility of using this approach to enhance human performance in tasks requiring faster reaction times, such as sports.

Comparing Humans and Large Language Models on an Experimental Protocol Inventory for Theory of Mind Evaluation (EPITOME)

Abstract We address a growing debate about the extent to which large language models (LLMs) produce behavior consistent with Theory of Mind (ToM) in humans. We present EPITOME: a battery of six experiments that tap diverse ToM capacities, including belief attribution, emotional inference, and pragmatic reasoning. We elicit a performance baseline from human participants for each task. We use the dataset to ask whether distributional linguistic information learned by LLMs is sufficient to explain ToM in humans. We compare performance of five LLMs to a baseline of responses from human comprehenders. Results are mixed. LLMs display considerable sensitivity to mental states and match human performance in several tasks. Yet, they commit systematic errors in others, especially those requiring pragmatic reasoning on the basis of mental state information. Such uneven performance indicates that human-level ToM may require resources beyond distributional information.

The Effect of Icon Color Combinations in Information Interfaces on Task Performance under Varying Levels of Cognitive Load

In human–computer interaction interfaces, icons serve as highly symbolic elements that convey information and significantly influence the performance of visual search and other tasks. Thus, the selection of appropriate colors plays a crucial role in the design of human–computer interfaces. This study aimed to investigate the effects of icon color combinations on human visual search task performance across various cognitive load conditions. The experiment was divided into two parts, involving a cognitive load test experiment and an icon search task, wherein the former required participants to select the target icon from 16 de-colored icons, and the latter involved selecting the target icon from 16 color combinations of 8 colors (background colors: black, red, blue, and purple; foreground colors: white, yellow, green, and turquoise). A total of 20 participants (11 females and 9 males, all aged between 18 and 24 years) were recruited for this experiment to perform both tasks sequentially. Through a comprehensive consideration of factors such as the NASA-TLX scale, retrieval time, etc., the experimental results revealed that cognitive load increased with time pressure, resulting in varying optimal color combinations for different cognitive load levels. Additionally, background colors did not affect task response time under different cognitive loads, whereas white foreground color was superior to turquoise foreground color in a medium cognitive load environment. Meanwhile, our results recommend prioritizing white-on-black as the preferred color combination, given that it demonstrated excellent cognitive performance in all three cognitive load environments. Conversely, white-on-blue is not recommended as a color combination for the design of high cognitive load environments. Concerning medium cognitive load environments, yellow-on-red or white-on-purple color combinations were preferred. Finally, in low cognitive load environments, white-on-purple is recommended as the primary color combination. Overall, this study provides a theoretical reference for the future design of interactive interface icons across various contexts.

Optimizing Human-Robot Teaming Performance through Q-Learning-Based Task Load Adjustment and Physiological Data Analysis.

The transition to Industry 4.0 and 5.0 underscores the need for integrating humans into manufacturing processes, shifting the focus towards customization and personalization rather than traditional mass production. However, human performance during task execution may vary. To ensure high human-robot teaming (HRT) performance, it is crucial to predict performance without negatively affecting task execution. Therefore, to predict performance indirectly, significant factors affecting human performance, such as engagement and task load (i.e., amount of cognitive, physical, and/or sensory resources required to perform a particular task), must be considered. Hence, we propose a framework to predict and maximize the HRT performance. For the prediction of task performance during the development phase, our methodology employs features extracted from physiological data as inputs. The labels for these predictions-categorized as accurate performance or inaccurate performance due to high/low task load-are meticulously crafted using a combination of the NASA TLX questionnaire, records of human performance in quality control tasks, and the application of Q-Learning to derive task-specific weights for the task load indices. This structured approach enables the deployment of our model to exclusively rely on physiological data for predicting performance, thereby achieving an accuracy rate of 95.45% in forecasting HRT performance. To maintain optimized HRT performance, this study further introduces a method of dynamically adjusting the robot's speed in the case of low performance. This strategic adjustment is designed to effectively balance the task load, thereby enhancing the efficiency of human-robot collaboration.

Artificial Creativity? Evaluating AI Against Human Performance in Creative Interpretation of Visual Stimuli

This study investigates the creative capacities of artificial intelligence (AI), exemplified by ChatGPT-4, in comparison with human creativity, utilizing the Figural Interpretation Quest (FIQ) as the evaluative tool. The primary objective is to assess whether AI can surpass human performance in creative tasks, particularly in terms of flexibility and subjectively perceived creativity. Participants, including both ChatGPT-4 and humans, were asked to provide creative interpretations for a set of ambiguous, abstract figures. The study measured the flexibility of these interpretations and gathered subjective ratings of creativity from independent evaluators. Results indicated that while AI, on average, demonstrated higher flexibility in generating creative interpretations, human participants excelled in terms of subjectively perceived creativity. Furthermore, the most creative human responses were higher than those of AI in both flexibility and subjective creativity. These findings suggest that in these types of complex multimodal interpretation tasks, AI shows excellent potential and the ability to generate semantically diverse ideas, yet human subjectively perceived creativity remains for now unmatched. The study highlights the limitations of AI in replicating the complexity of human creativity and points to possible theoretical interpretations of the reported findings.

Perceptual learning changes the amplitude not the shape of the temporal window of visual processing.

Perceptual learning (PL) can significantly improve human performance in perceptual tasks primarily through template reweighting. Previous studies have documented how PL changes perceptual template in stimulus feature space. We investigated how PL reweights visual information in time. With a dynamic external noise paradigm and the elaborated perceptual template model (ePTM) analysis, we found that training with an orientation identification task in the zero external noise condition reduced contrast thresholds in both zero and high external noise conditions, whereas training in the high external noise condition only reduced contrast thresholds in high external noise conditions. The ePTM analysis showed that training in both zero and high external noise changed the overall amplitude, but not the shape of the temporal window of the perceptual template to exclude external noise across time, and training in zero external noise additionally reduced additive internal noise. Our results provided additional constraints for models of PL. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Investigating effect of standardized total body skin examination using sequence-networks

This study employs sequence-network analysis to investigate the influence of instructing a standardized procedure on total-body skin examination (TBSE) performance. A between-subjects study was conducted with thirty-one participants comprising medical students and attending physicians. Among these participants, fifteen were randomly assigned to the uninstructed group and sixteen to the instructed group. The participants' gaze and field of view were recorded using eye tracking glasses while performing TBSE on a male and a female standardized patient. The recordings were then transcribed to depict the examination process. The instructed group missed significantly fewer body parts (p = 0.045) and had higher time efficiency (p = 0.007) while examining the female patient, but no significant difference was observed for the male patient examination. Furthermore, the examination sequences of the instructed group contained lesser variability than the uninstructed group. Hence, implementing a standard optimal procedure to perform TBSE could minimize the likelihood of missing body parts, increase examination efficiency, and improve performance consistency. This study demonstrated the potential of sequence-network analysis to study human performance in sequential tasks.

Performance evaluation of automated scoring for the descriptive similarity response task

We examined whether a machine-learning-based automated scoring system can mimic the human similarity task performance. We trained a bidirectional encoder representations from transformer-model based on the semantic similarity test (SST), which presented participants with a word pair and asked them to write about how the two concepts were similar. In Experiment 1, based on the fivefold cross validation, we showed the model trained on the combination of the responses (N = 1600) and classification criteria (which is the rubric of the SST; N = 616) scored the correct labels with 83% accuracy. In Experiment 2, using the test data obtained from different participants in different timing from Experiment 1, we showed the models trained on the responses alone and the combination of responses and classification criteria scored the correct labels in 80% accuracy. In addition, human–model scoring showed inter-rater reliability of 0.63, which was almost the same as that of human–human scoring (0.67 to 0.72). These results suggest that the machine learning model can reach human-level performance in scoring the Japanese version of the SST.

Automation function and malfunction: effects on human performance in accident handling tasks

By conducting a mixed-design experiment using simplified accident handling tasks performed by two-person teams, this study examined the effects of automation function and condition (before, during, and after malfunction) on human performance. Five different and non-overlapping functions related to human information processing model were considered and their malfunctions were set in a first-failure way. The results showed that while the automation malfunction impaired task performance, the performance degradation for information analysis was more severe than response planning. Contrary to other functions, the situation awareness for response planning and response implementation tended to increase during malfunctioning and decrease after. In addition, decreased task performance reduced trust in automation, and malfunctions in earlier stages of information processing resulted in lower trust. Suggestions provided for the design and training related to automation emphasise the importance of high-level cognitive support and the benefit of involving automation error handling in training.

Touching the Ground: Evaluating the Effectiveness of Data Physicalizations for Spatial Data Analysis Tasks.

Inspired by recent advances in digital fabrication, artists and scientists have demonstrated that physical data encodings (i.e., data physicalizations) can increase engagement with data, foster collaboration, and in some cases, improve data legibility and analysis relative to digital alternatives. However, prior empirical studies have only investigated abstract data encoded in physical form (e.g., laser cut bar charts) and not continuously sampled spatial data fields relevant to climate and medical science (e.g., heights, temperatures, densities, and velocities sampled on a spatial grid). This paper presents the design and results of the first study to characterize human performance in 3D spatial data analysis tasks across analogous physical and digital visualizations. Participants analyzed continuous spatial elevation data with three visualization modalities: (1) 2D digital visualization; (2) perspective-tracked, stereoscopic "fishtank" virtual reality; and (3) 3D printed data physicalization. Their tasks included tracing paths downhill, looking up spatial locations and comparing their relative heights, and identifying and reporting the minimum and maximum heights within certain spatial regions. As hypothesized, in most cases, participants performed the tasks just as well or better in the physical modality (based on time and error metrics). Additional results include an analysis of open-ended feedback from participants and discussion of implications for further research on the value of data physicalization. All data and supplemental materials are available at https://osf.io/7xdq4/.

How Time Pressure in Different Phases of Decision-Making Influences Human-AI Collaboration

Human cognitive and decision-making abilities depreciate under pressure, motivating the emergence of artificial intelligence (AI) systems as decision support tools to assist people in performing tasks under stress. In this work, we study human decision-making behavior and task performance under time pressure---induced from limitedinitial observation time (time to perform the task before providing an initial response without AI input) andfinal decision time (time to weigh an AI's suggestion before reaching a collective human-AI team answer)---for spatial reasoning and count estimation tasks. Our results show that, while the impact of initial observation time on AI-assisted decision-making was dependent on task nature, participants were more likely to follow AI suggestions when they were provided with longer final decision time; moreover, although participants generally tended to adhere to their initial responses, they had more agency when they were more logically engaged in a task. Our results offer a nuanced understanding of human-AI collaboration under time pressure in different phases of the decision-making process.

Personalized optimal room temperature and illuminance for maximizing occupant's mental task performance using physiological data

Indoor room temperature and illuminance level are critical factors of indoor environment quality (IEQ), affecting human mental task performance. These effects are reflected in their physiological responses such as heart rate, electrodermal activity, and skin temperature. Occupants' individual preferences, sensitivity, and physiological responses to different combinations of room temperature and illuminance level can differ among individuals. Despite previous studies investigating the individual and combined effects of different IEQ parameters, the limited research on the cross-modal relationship between room temperature and illuminance level and its impact on mental task performance highlights its significance. Moreover, to achieve personalized insights, it is essential to incorporate individual physiological responses, and this necessitates the development of an optimization model to comprehensively examine their impact. To address these issues, this study proposes a personalized model that optimizes room temperature and illuminance levels to enhance mental task performance using occupants' physiological data. Having the random forest algorithm, this study first predicted mental task performance, which includes four mental abilities such as attention, perception, working memory, and thinking ability using the occupant's physiological data. Then, the particle swarm optimization algorithm was employed to optimize room temperature and illuminance level to maximize the predicted mental task performance. The results of the proposed model align with observed values of room temperature and illuminance level during experiments, validating the adoption of a personalized approach. The findings contribute to future insights and guidelines for the design and management of indoor environments to maximize occupants' performance.

Explainable Deep Learning for False Information Identification: An Argumentation Theory Approach

To combat false information, social media sites have heavily relied on content moderation, mostly performed by human workers. However, human content moderation entails multiple problems, including huge labor costs, ineffectiveness, and ethical issues. To overcome these concerns, social media companies are aggressively investing in the development of artificial intelligence-powered false information detection systems. Extant efforts, however, have failed to understand the nature of human argumentation, delegating the process of making inferences of the truth to the black box of neural networks. They fail to attend to important aspects of how humans make judgments on the veracity of an argument, creating important challenges. To this end, based on Toulmin’s model of argumentation, we propose a computational framework that helps machine learning for false information identification understand the connection between a claim (whose veracity needs to be verified) and evidence (which contains information to support or refute the claim). The two experiments for testing model performance and explainability reveal that our framework shows stronger performance and better explainability, outperforming cutting-edge machine learning methods and presenting positive effects on human task performance, trust in algorithms, and confidence in decision making. Our results shed new light on the growing field of automated false information identification.

K-means Clustering Machine Learning Approach Reveals Groups of Homogeneous Individuals with Unique Brain Activation, Task, and Performance Dynamics using fNIRS.

Wearable functional near-infrared spectroscopy (fNIRS) for measuring brain function, in terms of hemodynamic responses, is pervading our everyday life and holds the potential to reliably classify cognitive load in a naturalistic environment. However, human's brain hemodynamic response, behavior, and cognitive and task performance vary, even within and across homogeneous individuals (with same training and skill sets), which limits the reliability of any predictive model for human. In the context of high-stakes tasks, such as in military and first-responder operations, the real-time monitoring of cognitive functions and relating it to the ongoing task, performance outcomes, and behavioral dynamics of the personnel and teams is invaluable. In this work, a portable wearable fNIRS system (WearLight) developed by the author was upgraded, and an experimental protocol was designed to image the prefrontal cortex (PFC) area of the brain of 25 healthy homogeneous participants in a naturalistic environment while participants performed n-back working memory (WM) tasks with four difficulty levels. The raw fNIRS signals were processed using a signal processing pipeline to derive the brain's hemodynamic responses. An unsupervised k-means machine learning (ML) clustering approach, utilizing the task-induced hemodynamic responses as input variables, suggested three unique participant groups. Task performance in terms of % correct, % missing, reaction time, inverse efficiency score (IES), and a proposed IES was extensively evaluated for each participant and the three groups. Results showed that, on average, brain hemodynamic response increased, whereas task performance degraded, with increasing WM load. However, the regression and correlation analysis of WM task, performance, and the brain's hemodynamic responses (TPH) revealedinteresting hidden characteristics and the variation in the TPH relationship between groups. The proposed IES also served as a better scoring method that had distinct score ranges for different load levels as opposed to the overlapping scores of the traditional IES method. Results showed that the k-means clustering has the potential to find groups of individuals in an unsupervised manner using the brain's hemodynamic responses and to study the underlying relationship between the TPH in groups. Using the method presented in this paper,real-time monitoring of cognitive and task performance of soldiers, and preferentially forming small units to accomplish tasks based on the insights and goals may be helpful. The results showed that WearLight can image PFC, and this study also suggests future directions for the multi-modal body sensor network (BSN) combining advanced ML algorithms for real-time state classification, cognitive and physical performance prediction, and the mitigation of performance degradation in the high-stakes environment.

Spike-based statistical learning explains human performance in non-adjacent dependency learning tasks

Grammar acquisition is of significant importance for mastering human language. As the language signal is sequential in its nature, it poses the challenging task to extract its structure during online processing. This modeling study shows how spike-timing dependent plasticity (STDP) successfully enables sequence learning of artificial grammars that include non-adjacent dependencies (NADs) and nested NADs. Spike-based statistical learning leads to synaptic representations that comply with human acquisition performances under various distributional stimulus conditions. STDP, therefore, represents a practicable neural mechanism underlying human statistical grammar learning. These findings highlight that initial stages of the language acquisition process are possibly based on associative learning strategies. Moreover, the applicability of STDP demonstrates that the non-human brain possesses potential precursor abilities that support the acquisition of linguistic structure.

Selection framework of visualization methods in designing AR industrial task-support systems

An Augmented reality (AR) task-support system incorporates head-mounted displays (HMDs) to improve human performance in maintenance, assembly, and disassembly tasks by intuitively showing the working procedure to the operator. To achieve the intended AR capability and to enable the operator to work efficiently, a system designer must understand the characteristics of AR and design the information to be shown operators for each subtask, taking into account the working situation represented by the procedure, operator performance skills, workspace, devices, and operating objects (i.e., AR information design). Especially, in the AR information design, the designer needs AR expertize to select the suitable visualization method among the many extant methods. This is difficult for designers (manual writers) to perform AR information design in factories in the future. However, at present, only a few studies have been conducted to select the suitable visualization method for each subtask. This study proposes to define 31 subtask types as criteria for decomposing tasks in AR information design. Furthermore, we classify and define 42 visualization methods in terms of AR information design. Finally, to select the suitable visualization method for each subtask, we construct a selection framework consisting of three selection categories representing the working status and 17 selection conditions. The evaluation experiments of the proposed framework could output an average of 4.3 suitable visualization method candidates, including at least one suitable visualization method selected by AR experts in the subtask working situation. By incorporating the proposed framework into a website and combining it with existing HMDs and services for AR application development, it is expected that designers will be able to develop appropriate AR task-support systems.

Social Inference from Relational Visual Information

Humans easily make social evaluations from visual input, such as recognizing a social interaction or deciding who is a friend and who is a foe. Prior efforts to model this ability have suggested that visual features and motion cues alone cannot account for human performance in social tasks, like distinguishing between helping and hindering interactions. On the other hand, generative Bayesian inference models, which make predictions based on simulations of agents’ social or physical goals, accurately predict human judgments, but are computationally very expensive and often impossible to implement in natural visual stimuli. Inspired by developmental work, we hypothesize that introducing inductive biases in visual models would allow them to make more human-like social judgments. Specifically, in this study, we investigate if relational representations of visual stimuli using graph neural networks can predict human social interaction judgments. We use the PHASE dataset, consisting of 2D animations of two agents and two objects resembling real-life social interactions, rated as friendly, neutral, or adversarial. We propose a graph neural networks (GNN) based architecture, that takes in graph representations of a video and predicts the relationship between the agents. We collected human ratings for each of the 400 videos and found that our GNN model aligns with human judgments significantly better than a baseline visual model with the same visual/motion information but without the graph structure (79% vs 62% prediction of human judgements). Intriguingly, explicitly adding relational information to the baseline model does not improve its performance, suggesting that graphical representations in particular are important to modeling human judgements. Taken together, these results suggest that relational graphical representations of visual information can help artificial vision systems make more human-like social judgments without incurring the computational cost of Bayesian models, and provide insights into the computations that humans employ while making visual social judgments.

Mitigating Threshold Effects in Human Control by Stochastic Resonance With Fractional Colored Noise

In industrial applications, mechanical and physiological thresholds may limit the capability of human manipulating machine via control devices, such as joysticks and steering wheels. These thresholds can result in loss of information in the control signals that are kept below the threshold of detection of the device or the human operator. One approach to mitigate these effects is <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">stochastic resonance</i> , i.e., by injecting additive noise into a signal to raise its energy content over the threshold of detection. Though this noise partially corrupts the signal, it can increase the detectability of the signal by the control device. This article provides, for the first time, research towards using <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">stochastic resonance</i> to improve human performance in control tasks. In particular, it shows that using adaptive colored noise can improve the detectability of the steering control signals recorded from human participants. The approach converts a signal processing task to an optimization problem, where <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">particle swarm optimization</i> is employed to obtain the optimal color (or spectral exponent) of the injected additive noise, generated through an intelligent technique with fractional order filters. The results have shown that the proposed method improves the detectability of subthreshold steering control signals. This method can be widely applicable to other industrial domains, such as energy harvesting and enhancing sensory perception.

Effects of Temporary Respiration Exercise with Individual Harmonic Frequency on Blood Pressure and Autonomic Balance

This study investigated the effects of modulated respiration on blood pressure and autonomic balance to develop a healthcare application system for stabilizing autonomic balance. Thirty-two participants were asked to perform self-regulated tasks with 18 different respiration sequences, and their electrocardiograms (ECG) and blood pressure were measured. Changes in cardiovascular system functions and blood pressure were compared between free-breathing and various respiration conditions. Systolic and diastolic blood pressures stabilized after individual harmonic breathing. Autonomic balance, characterized by heart rate variability, was also stabilized with brief respiration training according to harmonic frequency. Five machine-learning algorithms were used to classify the two opposing factors between the free and modulated breathing conditions. The random forest models outperformed the other classifiers in the training data of systolic blood pressure and heart rate variability. The mean areas under the curves (AUCs) were 0.82 for systolic blood pressure and 0.98 for heart rate variability. Our findings lend support that blood pressure and autonomic balance were improved by temporary harmonic frequency respiration. This study provides a self-regulated respiration system that can control and help stabilize blood pressure and autonomic balance, which would help reduce mental stress and enhance human task performance in various fields.

The effect of multi‐tasking training on performance, situation awareness, and workload in simulated air traffic control

AbstractIncreasingly higher demands are being made on the capacity‐limited cognitive capabilities of human operators as they strive to maintain situation awareness (i.e., understanding “what is going on”) and performance in complex tasks. In the current study we asked whether: (a) training administered via a mobile phone‐based app could improve multitasking and (b) improved multitasking in the app would generalize to improved performance and situation awareness in a simulated air traffic control task (ATC). Participants completed the ATC task before and after multiple sessions of app‐based multitasking training or control training. Multitasking on the app improved across training sessions. However, this did not lead to improved performance or situation awareness, or workload reduction, relative to control training on the ATC task. These outcomes indicate that app‐based multitasking training based on repetition of a single training task will not necessarily yield generalizable benefits to human performance in other complex dynamic tasks.