Effects of Display Response Latency on Brain Activity During Device Operation

Abstract

Human–machine interface (HMI) devices have become multifunctional and complicated to operate, requiring better usability. Nevertheless, response latency (time lag between operation input and display output) negatively impacts device usability because it is stressful and reduces the operator’s sense of agency (SoA)—the sense that the operator is the one who is causing or generating an action. Stress and SoA information can be obtained by measuring physiological indicators, such as brain activity. This study was designed to investigate the effects of response latency on the operator’s brain activity state through near-infrared spectroscopy, which measures the brain activity state based on changes in the concentration of oxygenated hemoglobin (ΔOxy–Hb). In this study, 15 subjects performed a target-tracking task in which they rotated an HMI commander used as an in-vehicle device to manipulate LED markers on a display. The temporal response latency (0, 50, 100, and 150 ms) between manipulating the commander and the display response was introduced, and the subject’s brain activity, subjective evaluation, and performance efficiency were measured. The results revealed that brain activity differed depending on whether the latency was recognized rather than the latency length, and that ΔOxy–Hb at the anterior prefrontal cortex of those who recognized the latency increased significantly, whereas that of those who did not recognize the latency did not increase. Specifically, latency perception activates the prefrontal cortex due to stress and the inability to prepare normally for the next action, and not recognizing latency inactivates the premotor cortex.