Pupil Diameter Data Research Articles

Using Pupil Diameter for Psychological Resilience Assessment in Medical Students Based on SVM and SHAP Model.

Effectively assessing psychological resilience for medical students is vital for identifying at-risk individuals and developing tailored interventions. At present, few studies have combined physiological indexes of the human body and machine learning for psychological resilience assessment. This study presents a novel approach that employs pupil diameter features and machine learning to predict psychological resilience risk objectively. Firstly, we designed a stimulus paradigm (via auditory and visual stimuli) and collected pupil diameter data from participants using eye-tracking technology. Secondly, the pupil data was preprocessed, including linear interpolation, blink detection, and subtractive baseline correction. Thirdly, statistical metrics were extracted and optimal feature subsets were obtained by Recursive Feature Elimination with Cross-Validation (RFECV). Subsequently, the classification models, including Logistic Regression (LR), Random Forest (RF), Support Vector Machine (SVM), and eXtreme Gradient Boosting (XGBoost), were trained. The experimental results show that the SVM model has the best performance, and its balance accuracy, recall, and AUC reach 0.906, 0.89, and 0.932, respectively. Finally, we leveraged the Shapley additive explanation (SHAP) model for interpretability analysis. It revealed auditory stimuli have a more significant effect than visual stimuli in psychological resilience assessment. These findings suggested that pupil diameter could be a vital metric for assessing psychological resilience.

Extended reality quantification of pupil reactivity as a non-invasive assessment for the pathogenesis of spaceflight associated neuro-ocular syndrome: A technology validation study for astronaut health

The National Aeronautics and Space Administration (NASA) has rigorously documented a group of neuro-ophthalmic findings in astronauts during and after long-duration spaceflight known as spaceflight associated neuro-ocular syndrome (SANS). For astronaut safety and mission effectiveness, understanding SANS and countermeasure development are of utmost importance. Although the pathogenesis of SANS is not well defined, a leading hypothesis is that SANS might relate to a sub-clinical increased intracranial pressure (ICP) from cephalad fluid shifts in microgravity. However, no direct ICP measurements are available during spaceflight. To further understand the role of ICP in SANS, pupillometry can serve as a promising non-invasive biomarker for spaceflight environment as ICP is correlated with the pupil variables under illumination. Extended reality (XR) can help to address certain limitations in current methods for efficient pupil testing during spaceflight. We designed a protocol to quantify parameters of pupil reactivity in XR with an equivalent time duration of illumination on each eye compared to pre-existing, non-XR methods. Throughout the assessment, the pupil diameter data was collected using HTC Vive Pro-VR headset, thanks to its eye-tracking capabilities. Finally, the data was used to compute several pupil variables. We applied our methods to 36 control subjects. Pupil variables such as maximum and minimum pupil size, constriction amplitude, average constriction amplitude, maximum constriction velocity, latency and dilation velocity were computed for each control data. We compared our methods of calculation of pupil variables with the non-XR methods existing in the literature. Distributions of the pupil variables such as latency, constriction amplitude, and velocity of 36 control data displayed near-identical results from the non-XR literature for normal subjects. We propose a new method to evaluate pupil reactivity with XR technology to further understand ICP's role in SANS and provide further insight into SANS countermeasure development for future spaceflight.

Wavelet-based approach for diagnosing attention deficit hyperactivity disorder (ADHD)

Attention deficit hyperactivity disorder (ADHD) is a common cognitive disorder affecting children. ADHD can interfere with educational, social, and emotional development, so early detection is essential for obtaining proper care. Standard ADHD diagnostic protocols rely heavily on subjective assessments of perceived behavior. An objective diagnostic measure would be a welcome development and potentially aid in accurately and efficiently diagnosing ADHD. Analysis of pupillary dynamics has been proposed as a promising alternative method of detecting affected individuals effectively. This study proposes a method based on the self-similarity of pupillary dynamics and assesses its strength as a potential diagnostic biomarker. Localized discriminatory features are developed in the wavelet domain and selected via a rolling window method to build classifiers. The application on a task-based pupil diameter time series dataset of children aged 10–12 years shows that the proposed method achieves greater than 78% accuracy in detecting ADHD. Comparing with a recent approach that constructs features in the original data domain, the proposed wavelet-based classifier achieves more accurate ADHD classification with fewer features. The findings suggest that the proposed diagnostic procedure involving interpretable wavelet-based self-similarity features of pupil diameter data can potentially aid in improving the efficacy of ADHD diagnosis.

Stateful-Service-Based Pupil Recognition in Natural Light Environments.

Smartphones are currently extensively used worldwide, and advances in hardware quality have engendered improvements in smartphone image quality, which is occasionally comparable to the quality of medical imaging systems. This paper proposes two algorithms for pupil recognition: a stateful-service-based pupil recognition mechanism and color component low-pass filtering algorithm. The PRSSM algorithm can determine pupil diameters in images captured in indoor natural light environments, and the CCLPF algorithm can determine pupil diameters in those captured outdoors under sunlight. The PRSSM algorithm converts RGB colors into the hue saturation value color space and performs adaptive thresholding, morphological operations, and contour detection for effectively analyzing the diameter of the pupil. The CCLPF algorithm derives the average matrix for the red components of eye images captured in outdoor environments. It also performs low-pass filtering, morphological and contour detection operations, and rule-of-thumb correction. This algorithm can effectively analyze pupil diameter in outdoor natural light. Traditional ruler-based measurements of pupil diameter were used as the reference to verify the accuracy of the PRSSM and CCLPF algorithms and to compare their accuracy with that of the other algorithm. The errors in pupil diameter data were smaller for the PRSSM and CCLPF algorithms than for the other algorithm.

Analyzing the Influencing Factors and Workload Variation of Takeover Behavior in Semi-Autonomous Vehicles.

There are many factors that will influence the workload of drivers during autonomous driving. To examine the correlation between different factors and the workload of drivers, the influence of different factors on the workload variations is investigated from subjective and objective viewpoints. Thirty-seven drivers were recruited to participant the semi-autonomous driving experiments, and the drivers were required to complete different NDRTs (Non-Driving-Related Tasks): mistake finding, chatting, texting, and monitoring when the vehicle is in autonomous mode. Then, we introduced collision warning to signal there is risk ahead, and the warning signal was triggered at different TB (Time Budget)s before the risk, at which time the driver had to take over the driving task. During driving, the NASA-TLX-scale data were obtained to analyze the variation of the driver’s subjective workload. The driver’s pupil-diameter data acquired by the eye tracker from 100 s before the TOR (Take-Over Request) to 19 s after the takeover were obtained as well. The sliding time window was set to process the pupil-diameter data, and the 119-s normalized average pupil-diameter data under different NDRTs were fitted and modeled to analyze the variation of the driver’s objective workload. The results show that the total subjective workload score under the influence of different factors is as follows: obstacle-avoidance scene > lane-keeping scene; TB = 7 s and TB = 3 s have no significant difference; and mistake finding > chatting > texting > monitoring. The results of pupil-diameter data under different factors are as follows: obstacle-avoidance scene > lane-keeping scene; TB = 7 s > TB = 3 s; and monitoring type (chatting and monitoring) > texting type (mistake finding and texting). The research results can provide a reference for takeover safety prediction modeling based on workload.

Superior frontal regions reflect the dynamics of task engagement and theta band-related control processes in time-on task effects

Impairment of cognitive performance is often observed in time-on tasks. Theoretical considerations suggest that especially prefrontal cortex cognitive control functions is affected by time-on-task effects, but the role of effort/task engagement is not understood. We examine time-on-task effects in cognitive control on a neurophysiological level using a working-memory modulated response inhibition task and inter-relate prefrontal neuroanatomical region-specific theta-band activity with pupil diameter data using EEG-beamforming approaches. We show that task performance declines with time-on tasks, which was paralleled by a concomitant decreases of task-evoked superior frontal gyrus theta-band activity and a reduction in phasic pupil diameter modulations. A strong relation between cognitive control-related superior frontal theta-band activity and effort/task engagement indexed by phasic pupil diameter modulations was observed in the beginning of the experiment, especially for tasks requiring inhibitory controls and demanding high working memory. This strong relation vanished at the end of the experiment, suggesting a decoupling of cognitive control resources useable for a task and effort invested that characterizes time-on-task effects in prefrontal cortical structures.

A role of the norepinephrine system or effort in the interplay of different facets of inhibitory control

Inhibitory control has multiple facets, and one possible distinction can be made between ‘inhibition of interferences’ and the ‘inhibition of actions’. Both facets of inhibitory control show an interdependency. Even though some neurophysiological processes underlying this interdependency have been examined, the role of neuro-modulatory processes in their interplay are not understood. In the current study, we examine the role of the norepinephrine (NE) system in these processes. We did so by combining a Go/Nogo and Simon task. We recorded the EEG and pupil diameter data as an indirect index of NE system activity during the task. EEG theta band activity data and pupil diameter data were then integrated after conducting a temporal signal decomposition of the EEG data. We show that particularly theta band activity coding stimulus-response translation processes associated with middle frontal cortices, but not stimulus-driven processes are modulated by the interplay between the ‘inhibition of interferences’ and the ‘inhibition of actions’. Modulations in stimulus-response translation processes were systematically correlated with pupil-diameter responses. The pattern of correlations suggests that phasic NE system activity particularly modulates stimulus-response mapping processes during conflict monitoring in incongruent Nogo trials, which may explain behavioral performance effects. Phasic NE system activity reflects essential modulators of the interplay between the ‘inhibition of interferences’ and the ‘inhibition of actions’.

Time-On-Task Effects on Working Memory Gating Processes-A Role of Theta Synchronization and the Norepinephrine System.

Performance impairment as an effect of prolonged engagement in a specific task is commonly observed. Although this is a well-known effect in everyday life, little is known about how this affects central cognitive functions such as working memory (WM) processes. In the current study, we ask how time-on-task affects WM gating processes and thus processes regulating WM maintenance and updating. To this end, we combined electroencephalography methods and recordings of the pupil diameter as an indirect of the norepinephrine (NE) system activity. Our results showed that only WM gate opening but not closing processes showed time-on-task effects. On the neurophysiological level, this was associated with modulation of dorsolateral prefrontal theta band synchronization processes, which vanished with time-on-task during WM gate opening. Interestingly, also the modulatory pattern of the NE system, as inferred using pupil diameter data, changed. At the beginning, a strong correlation of pupil diameter data and theta band synchronization processes during WM gate opening is observed. This modulatory effect vanished at the end of the experiment. The results show that time-on-task has very specific effects on WM gate opening and closing processes and suggests an important role of NE system in the time-on-task effect on WM gate opening process.

Assessment method of depressive disorder level based on graph attention network

This paper presents an approach to predict the depression self-rating scale of Patient Health Questions-9 (PHQ-9) values from pupil-diameter data based on the graph attention network (GAT). The pupil diameter signal was derived from the eye information collected synchronously while the subjects were viewing the virtual reality emotional scene, and then the scores of PHQ-9 depression self-rating scale were collected for depression level. The chebyshev distance based GAT (Chebyshev-GAT) was constructed by extracting pupil-diameter change rate, emotional bandwidth, information entropy and energy, and their statistical distribution. The results show that, the error (MAE and SMRE)of the prediction results using Chebyshev-GAT is smaller then the traditional regression prediction model.

Machine learning-based analysis of operator pupillary response to assess cognitive workload in clinical ultrasound imaging

IntroductionPupillometry, the measurement of eye pupil diameter, is a well-established and objective modality correlated with cognitive workload. In this paper, we analyse the pupillary response of ultrasound imaging operators to assess their cognitive workload, captured while they undertake routine fetal ultrasound examinations. Our experiments and analysis are performed on real-world datasets obtained using remote eye-tracking under natural clinical environmental conditions. MethodsOur analysis pipeline involves careful temporal sequence (time-series) extraction by retrospectively matching the pupil diameter data with tasks captured in the corresponding ultrasound scan video in a multi-modal data acquisition setup. This is followed by the pupil diameter pre-processing and the calculation of pupillary response sequences. Exploratory statistical analysis of the operator pupillary responses and comparisons of the distributions between ultrasonographic tasks (fetal heart versus fetal brain) and operator expertise (newly-qualified versus experienced operators) are performed. Machine learning is explored to automatically classify the temporal sequences into the corresponding ultrasonographic tasks and operator experience using temporal, spectral, and time-frequency features with classical (shallow) models, and convolutional neural networks as deep learning models. ResultsPreliminary statistical analysis of the extracted pupillary response shows a significant variation for different ultrasonographic tasks and operator expertise, suggesting different extents of cognitive workload in each case, as measured by pupillometry. The best-performing machine learning models achieve receiver operating characteristic (ROC) area under curve (AUC) values of 0.98 and 0.80, for ultrasonographic task classification and operator experience classification, respectively. ConclusionWe conclude that we can successfully assess cognitive workload from pupil diameter changes measured while ultrasound operators perform routine scans. The machine learning allows the discrimination of the undertaken ultrasonographic tasks and scanning expertise using the pupillary response sequences as an index of the operators’ cognitive workload. A high cognitive workload can reduce operator efficiency and constrain their decision-making, hence, the ability to objectively assess cognitive workload is a first step towards understanding these effects on operator performance in biomedical applications such as medical imaging.

The Relationship Between Pupil Diameter Data and Confidence in Multi-Stage Decisions

The Relationship Between Pupil Diameter Data and Confidence in Multi-Stage Decisions

Four-class emotion classification in virtual reality using pupillometry

BackgroundEmotion classification remains a challenging problem in affective computing. The large majority of emotion classification studies rely on electroencephalography (EEG) and/or electrocardiography (ECG) signals and only classifies the emotions into two or three classes. Moreover, the stimuli used in most emotion classification studies utilize either music or visual stimuli that are presented through conventional displays such as computer display screens or television screens. This study reports on a novel approach to recognizing emotions using pupillometry alone in the form of pupil diameter data to classify emotions into four distinct classes according to Russell’s Circumplex Model of Emotions, utilizing emotional stimuli that are presented in a virtual reality (VR) environment. The stimuli used in this experiment are 360° videos presented using a VR headset. Using an eye-tracker, pupil diameter is acquired as the sole classification feature. Three classifiers were used for the emotion classification which are Support Vector Machine (SVM), k-Nearest Neighbor (KNN), and Random Forest (RF).FindingsSVM achieved the best performance for the four-class intra-subject classification task at an average of 57.05% accuracy, which is more than twice the accuracy of a random classifier. Although the accuracy can still be significantly improved, this study reports on the first systematic study on the use of eye-tracking data alone without any other supplementary sensor modalities to perform human emotion classification and demonstrates that even with a single feature of pupil diameter alone, emotions could be classified into four distinct classes to a certain level of accuracy. Moreover, the best performance for recognizing a particular class was 70.83%, which was achieved by the KNN classifier for Quadrant 3 emotions.ConclusionThis study presents the first systematic investigation on the use of pupillometry as the sole feature to classify emotions into four distinct classes using VR stimuli. The ability to conduct emotion classification using pupil data alone represents a promising new approach to affective computing as new applications could be developed using readily-available webcams on laptops and other mobile devices that are equipped with cameras without the need for specialized and costly equipment such as EEG and/or ECG as the sensor modality.

A possible role of the norepinephrine system during sequential cognitive flexibility – Evidence from EEG and pupil diameter data

Sequential cognitive flexibility is a major requirement for goal-directed behavior. Recent findings show that inhibitory control processes are crucial for sequential cognitive flexibility. These processes are indicated by the ‘backward inhibition (BI) effect’ which emerges when a mental representation that has recently been suppressed in favour of another task has to be re-activated. Alterations in the catecholaminergic neural transmission including the norepinephrine (NE) system have been shown to modulate inhibitory processes. However, a possible role of the NE system in sequential cognitive flexibility is elusive. The present study examines the relevance of the NE system for sequential cognitive flexibility by integrating pupil diameter data and electrophysiological (EEG) data applying signal decomposition techniques and source localization. We show that the BI effects modulated amplitudes in the P1/N1 time window, as well as in the N2 time window. Correlating this data with the pupil diameter data only revealed substantial correlations in the P1/N1 time window. Moreover, it is shown that regions in the right inferior frontal gyrus are activated during modulations in the P1 time window, in which correlations with the pupil diameter data were also evident. The results are interpreted that sequential cognitive flexibility modulates early inhibitory gating processes (P1) which are related to the suppression of task-irrelevant information in inferior frontal regions. These processes are likely modulated by the NE system.

Properties of lower level processing modulate the actions of the norepinephrine system during response inhibition

We ask whether actions of the norepinephrine (NE) system during response inhibition are affected by properties of lower level sensory stimulus processing. We used a somato-sensory Go/Nogo task and combined ERP recordings with pupil diameter recordings as an index of NE system activity. The Go/Nogo task was designed to achieve processing of tactile stimuli predominantly over primary somatosensory (SI) and secondary somatosensory (SII) areas. The data show that response inhibition was better when stimuli were processed via SII, compared to SI areas. This was reflected by variations in the Nogo-N2/P3 associated with anterior cingulate structures. Correlations with the pupil diameter data, indicting modulations of the NE system during inhibitory control processes, were only evident when SI sensory areas were involved. These dissociable modulatory effects were associated with activations in the superior frontal gyrus. Actions of the NE system during response inhibition are modulated by properties of lower level processing.

Improving pupil diameter measurement accuracy in a remote eye tracking system

Pupil diameter (PD) has been used to track changes in mental effort across a broad range of cognitive tasks for over 60 years. PD is often measured from remote eye tracking systems, which all have the same limitation: the lack of a known reference value to convert the pixels captured within the systems to millimeters. Researchers frequently normalize their data within an individual to overcome this issue, however recent studies have found individual differences in resting PDs. This paper investigated the use of a fiduciary marker of a fixed size and an individual’s interpupillary distance, as known reference values. Both techniques substantially improved the accuracy of PD data compared to the unadjusted system data. Further, the average difference between both techniques and the uncorrected pupil diameter was just under .4mm, which is approximately the equivalent of most studies finding differences in cognitive load.

Looking through blue glasses: bioelectrical measures to assess the awakening after a calm situation.

Colors can elicit cognitive and emotional states. In particular, blue colour is associated to "refresh" and "restart" effects and is suggested to enhance a wake-up after a calm situation. In this exploratory study, these claims are investigated using Electroencephalographic (EEG), Skin Conductance (SC) and pupil diameter data. The results confirmed the "wake-up effect" for subjects wearing the lenses, as measured by Global Field Power (GFP) in Theta Band, Skin Conductance Response (SCR) and pupil diameter data.

How the depth of processing modulates emotional interference - evidence from EEG and pupil diameter data.

The ability to process emotionally conflicting information is an important requirement for emotional self-control. While it seems obvious that the impact of interfering emotional information critically depends on how deeply this interfering information is processed, it is still unknown what cognitive subprocesses are most affected by manipulating the depth of processing of emotionally interfering information. We examine these aspects integrating neurophysiological (EEG) and source localization data with pupil diameter data as an indirect index of the norepinephrine (NE) system activity. We show that when processing depth of interfering emotional stimulus dimensions is increased, emotional Stroop effects become stronger. The EEG data show that this was associated with modulations of decision-making processes, as reflected by the P3 event-related potential. Notably, the integration with pupil diameter data suggests that these decision processes were modulated by the NE system, especially when the depth of processing of interfering emotional stimulus dimensions was increased. This likely reflects gain modulation processes to facilitate processing of complex interfering, emotional information. The source localization results suggest that regions in the parietal (BA7) and insular cortex (BA13) are associated with these modulatory effects. The results suggest that overcoming more complex emotional interference triggers engagement of the norepinephrine system (indexed by pupil diameter) to facilitate action control mechanisms in a time-specific manner when deeper processing of emotional stimulus dimensions is required.

On the interrelation of 1/f neural noise and norepinephrine system activity during motor response inhibition.

Several lines of evidence suggest that there is a close interrelation between the degree of noise in neural circuits and the activity of the norepinephrine (NE) system, yet the precise nexus between these aspects is far from being understood during human information processing and cognitive control in particular. We examine this nexus during response inhibition in n = 47 healthy participants. Using high-density EEG recordings, we estimate neural noise by calculating "1/f noise" of those data and integrate these EEG parameters with pupil diameter data as an established indirect index of NE system activity. We show that neural noise is reduced when cognitive control processes to inhibit a prepotent/automated response are exerted. These neural noise variations were confined to the theta frequency band, which has also been shown to play a central role during response inhibition and cognitive control. There were strong positive correlations between the 1/f neural noise parameter and the pupil diameter data within the first 250 ms after the Nogo stimulus presentation at centro-parietal electrode sites. No such correlations were evident during automated responding on Go trials. Source localization analyses using standardized low-resolution brain electromagnetic tomography show that inferior parietal areas are activated in this time period in Nogo trials. The data suggest an interrelation of NE system activity and neural noise within early stages of information processing associated with inferior parietal areas when cognitive control processes are required. The data provide the first direct evidence for the nexus between NE system activity and the modulation of neural noise during inhibitory control in humans. NEW & NOTEWORTHY This is the first study showing that there is a nexus between norepinephrine system activity and the modulation of neural noise or scale-free neural activity during inhibitory control in humans. It does so by integrating pupil diameter data with analysis of EEG neural noise.

Anodal tDCS affects neuromodulatory effects of the norepinephrine system on superior frontal theta activity during response inhibition.

Medial and superior frontal theta oscillations are important for response inhibition. The norepinephrine (NE) system has been shown to modulate these oscillations possibly via gain control mechanisms, which depend on the modulation of neuron membrane potentials. Because the latter are also modulated by tDCS, the interrelation of tDCS and NE effects on superior frontal theta band activity needs investigation. We test the hypothesis that anodal tDCS affects modulatory effects of the NE system on theta band activity during inhibitory control in superior frontal regions. Using EEG beamforming, theta band activity in the superior frontal gyrus (SFG) was integrated (correlated) with the pupil diameter data as an indirect index of NE activity. In a within-subject design, healthy participants completed a response inhibition task in two sessions in which they received 2mA anodal tDCS over the vertex, or sham stimulation. There were no behavioral effects of anodal tDCS. Yet, tDCS affected correlations between SFG theta band activity time course and the pupil diameter time course. Correlations were evident after sham stimulation (r = .701; p < .004), but absent after anodal tDCS. The observed power of this dissociation was above 95%. The data suggest that anodal tDCS may eliminate neuromodulatory effects, likely of the NE system, on theta band activity during response inhibition in a structure of the response inhibition network. The NE system and tDCS seem to target similar mechanisms important for cognitive control in the prefrontal cortex. The results provide a hint why tDCS often fails to induce overt behavioral effects and shows that neurobiological systems, which may exert similar effects as tDCS on neural processes should closely be monitored in tDCS experiments.

Pupillary dilation response reflects surprising moments in music.

There are indications that the pupillary dilation response (PDR) reflects surprising moments in an auditory sequence such as the appearance of a deviant noise against repetitively presented pure tones (4), and salient and loud sounds that are evaluated by human paricipants subjectively (12). In the current study, we further examined whether the reflection of PDR in auditory surprise can be accumulated and revealed in complex and yet structured auditory stimuli, i.e., music, and when the surprise is defined subjectively. Participants listened to 15 excerpts of music while their pupillary responses were recorded. In the surprise-rating session, participants rated how surprising an instance in the excerpt was, i.e., rich in variation versus monotonous, while they listened to it. In the passive-listening session, they listened to the same 15 excerpts again but were not involved in any task. The pupil diameter data obtained from both sessions were time-aligned to the rating data obtained from the surprise-rating session. Results showed that in both sessions, mean pupil diameter was larger at moments rated more surprising than unsurprising. The result suggests that the PDR reflects surprise in music automatically.