Eye Movement Videos Research Articles

Agreement between vestibular elective physical therapy students and experienced physical therapists in identifying nystagmus during positional testing and diagnosing benign paroxysmal positional vertigo (BPPV)

Purpose: The aim of this study is to measure the effectiveness of a vestibular elective in preparing students to perform nystagmus identification and diagnosing benign paroxysmal positional vertigo (BPPV) through an investigation of agreement between student performances and experienced physical therapists. Methods: Seven students watched eye movement videos during positioning testing of 14 patients. Students recorded the nystagmus characteristics for each position and made diagnoses. The students completed additional surveys. Pairwise percent agreement and kappa statistics (κ) were used to compare agreement between each students’ conclusions to those of an experienced physical therapist obtained from a previous study. Krippendorff’s alpha (α) was used to analyze the overall reliability of student ability for these skills. Results: Interrater agreement between students and the expert was fair to almost perfect (κ = 0.22 to 1) in identifying nystagmus, and slight to almost perfect (κ = −0.02 to 1) in diagnosing BPPV. There was moderate agreement between students to identify nystagmus (α = 0.58–0.65, average agreement = 80–83%) during positional testing, and fair to moderate agreement to diagnose BPPV (α = 0.38 to 0.53, average agreement = 64–74%). Agreement was higher in cases of typical BPPV presentations. Students reported the task as difficult, had the necessary knowledge, and wanted more practice. Correlations (r = 0.76 to 0.82) were seen between clinical experience with BPPV and lower perceived difficulty. Conclusion: Students are capable of identifying nystagmus and diagnosing typical BPPV; however, students may require more practice, clinical experience, and mentorship to improve reliability. The results may help determine educational needs to prepare clinicians to manage individuals with BPPV.

Multimodal deep learning-based diagnostic model for BPPV

BackgroundBenign paroxysmal positional vertigo (BPPV) is a prevalent form of vertigo that necessitates a skilled physician to diagnose by observing the nystagmus and vertigo resulting from specific changes in the patient’s position. In this study, we aim to explore the integration of eye movement video and position information for BPPV diagnosis and apply artificial intelligence (AI) methods to improve the accuracy of BPPV diagnosis.MethodsWe collected eye movement video and diagnostic data from 518 patients with BPPV who visited the hospital for examination from January to March 2021 and developed a BPPV dataset. Based on the characteristics of the dataset, we propose a multimodal deep learning diagnostic model, which combines a video understanding model, self-encoder, and cross-attention mechanism structure.ResultOur validation test on the test set showed that the average accuracy of the model reached 81.7%, demonstrating the effectiveness of the proposed multimodal deep learning method for BPPV diagnosis. Furthermore, our study highlights the significance of combining head position information and eye movement information in BPPV diagnosis. We also found that postural and eye movement information plays a critical role in the diagnosis of BPPV, as demonstrated by exploring the necessity of postural information for the diagnostic model and the contribution of cross-attention mechanisms to the fusion of postural and oculomotor information. Our results underscore the potential of AI-based methods for improving the accuracy of BPPV diagnosis and the importance of considering both postural and oculomotor information in BPPV diagnosis.

Feasibility of video-based real-time nystagmus tracking: a lightweight deep learning model approach using ocular object segmentation.

Eye movement tests remain significantly underutilized in emergency departments and primary healthcare units, despite their superior diagnostic sensitivity compared to neuroimaging modalities for the differential diagnosis of acute vertigo. This underutilization may be attributed to a potential lack of awareness regarding these tests and the absence of appropriate tools for detecting nystagmus. This study aimed to develop a nystagmus measurement algorithm using a lightweight deep-learning model that recognizes the ocular regions. The deep learning model was used to segment the eye regions, detect blinking, and determine the pupil center. The model was trained using images extracted from video clips of a clinical battery of eye movement tests and synthesized images reproducing real eye movement scenarios using virtual reality. Each eye image was annotated with segmentation masks of the sclera, iris, and pupil, with gaze vectors of the pupil center for eye tracking. We conducted a comprehensive evaluation of model performance and its execution speeds in comparison to various alternative models using metrics that are suitable for the tasks. The mean Intersection over Union values of the segmentation model ranged from 0.90 to 0.97 for different classes (sclera, iris, and pupil) across types of images (synthetic vs. real-world images). Additionally, the mean absolute error for eye tracking was 0.595 for real-world data and the F1 score for blink detection was ≥ 0.95, which indicates our model is performing at a very high level of accuracy. Execution speed was also the most rapid for ocular object segmentation under the same hardware condition as compared to alternative models. The prediction for horizontal and vertical nystagmus in real eye movement video revealed high accuracy with a strong correlation between the observed and predicted values (r = 0.9949 for horizontal and r = 0.9950 for vertical; both p < 0.05). The potential of our model, which can automatically segment ocular regions and track nystagmus in real time from eye movement videos, holds significant promise for emergency settings or remote intervention within the field of neurotology.

Stratification of patients with Menière's disease based on eye movement videos recorded from the beginning of vertigo attacks and contrast-enhanced MRI findings.

Diagnosis of Menière's disease (MD) relies on subjective factors and the patients diagnosed with MD may have heterogeneous pathophysiologies. This study aims to stratify MD patients using two objective data, nystagmus videos and contrast-enhanced magnetic resonance imaging (CE-MRI). This is a retrospective cross-sectional study. According to the Japan Society for Equilibrium Research criteria (c-JSER), adults diagnosed with definite MD and who obtained videos recorded by portable nystagmus recorder immediately following vertigo attacks and underwent CE-MRI of the inner ear were included (ss = 91). Patients who obtained no nystagmus videos, who had undergone sac surgery, and those with long examination intervals were excluded (n = 40). The gender of the subjects was 22 males and 29 females. The age range was 20-82 y, with a median of 54 y. Endolymphatic hydrops (EH) were observed on CE-MRI in 84% (43 patients). Thirty-one patients had unilateral EH. All of them demonstrated EH on the side of the presence of cochlear symptoms. The number of patients who had both nystagmus and EH was 38. Five patients only showed EH and 5 patients only exhibited nystagmus, while 3 patients did not have either. Of the 43 nystagmus records, 32 showed irritative nystagmus immediately after the vertigo episode. The direction of nystagmus later reversed in 44% of cases over 24 h. Patients were stratified into subgroups based on the presence or absence of EH and nystagmus. The side with cochlear symptoms was consistent with EH. The c-JSER allows for the diagnosis of early-stage MD patients, and it can be used to treat early MD and preserve hearing; however, this approach may also include patients with different pathologies.

“Strategies are more important than words:” A case study of adult English Learners’ disciplinary reading

This case study explored adult Chinese-Mandarin-speaking English learners’ reading for academic purposes. Drawing on transactional socio-psycholinguistic perspectives, the researchers analyzed English learners’ reading processes through retrospective eye movement miscue analysis. Six international graduate students studying social science and engineering read selected academic texts in their disciplines, retold the texts, watched their eye movement videos, and discussed their miscues and eye movements. In addition, the researchers interviewed participants about their reading perspectives and strategy uses in their respective disciplinary areas. The findings showed that all participants read for meaning and learning new knowledge. Participants’ previous instruction and current disciplines both impacted their academic reading. They used general reading strategies and specific disciplinary strategies to read in their areas. The findings suggest that participants used all cueing systems and perceived academic reading as learning the content more than the language. Social science participants valued strategies and constantly reflected to improve their learning content knowledge. In contrast, engineering participants better identified their miscues and tended to correct for accuracy. Overall, the retrospective conversations helped them develop metacognition, understand their reading processes, and discover their strategies. This study offers educational implications for supporting post-secondary English learners’ reading for academic purposes.

Building a mental toolbox: Relationships between strategy choice and sight-singing performance in higher education.

Sight-singing is an inescapable component of music training in higher education and is often challenging for students. However, some strategies could help students perform. Yet, the extent to which students can use strategies to improve their sight-singing performance remains unclear. This article asks two questions to fill this gap: (1) Which strategies do students use when sight-singing? (2) Does the application of some types of strategy predict performance? We recruited 56 postsecondary music students and asked them about their musical backgrounds. They then sight-sang a short melody while we recorded their eye movements. After that, we conducted semi-structured retrospective interviews, using eye-movement videos and attention distribution heatmaps to help participants remember the strategies they used. We analyzed the interview transcripts to identify the strategies students used and regrouped them into categories. We extracted seven categories and discovered that using body movements predicted rhythm scores, that using musical knowledge predicted pitch and combined scores, and that relying on automatic skills predicted all dimensions of sight-singing performance. We recommend that aural skills instructors teach strategies explicitly and help students develop robust musical knowledge, as they are required to build strong automatic skills.

“My Eyes Move Dynamically”: Inquiring into Adult English Learners’ Reading Through Retrospective Eye Movement Miscue Analysis

AbstractThis paper investigated two adult English learners’ reading processes of reading informational texts through Retrospective Eye Movement Miscue Analysis (REMMA). The readers read two texts and retold what they read. Their reading was recorded by the Tobii Pro eye tracker. They watched their eye movement video and discussed their miscues and eye movements. The REMMA sessions helped the readers realize their misconception about reading, understand reading is meaning‐making, uncover their strategies, and grow confidence. When readers have illusions about reading, secondary teachers and adult educators could invite them to share their reading views, talk about their reading, and discuss the importance of using images in informational texts.

Nystagmus Estimation for Dizziness Diagnosis by Pupil Detection and Tracking Using Mexican-Hat-Type Ellipse Pattern Matching.

The detection of nystagmus using video oculography experiences accuracy problems when patients who complain of dizziness have difficulty in fully opening their eyes. Pupil detection and tracking in this condition affect the accuracy of the nystagmus waveform. In this research, we design a pupil detection method using a pattern matching approach that approximates the pupil using a Mexican hat-type ellipse pattern, in order to deal with the aforementioned problem. We evaluate the performance of the proposed method, in comparison with that of a conventional Hough transform method, for eye movement videos retrieved from Gifu University Hospital. The performance results show that the proposed method can detect and track the pupil position, even when only 20% of the pupil is visible. In comparison, the conventional Hough transform only indicates good performance when 90% of the pupil is visible. We also evaluate the proposed method using the Labelled Pupil in the Wild (LPW) data set. The results show that the proposed method has an accuracy of 1.47, as evaluated using the Mean Square Error (MSE), which is much lower than that of the conventional Hough transform method, with an MSE of 9.53. We conduct expert validation by consulting three medical specialists regarding the nystagmus waveform. The medical specialists agreed that the waveform can be evaluated clinically, without contradicting their diagnoses.

Deep learning based torsional nystagmus detection for dizziness and vertigo diagnosis

Dizziness and vertigo are common clinical symptoms and typical complaints of many vestibular diseases. In the bedside examination of dizziness and vertigo, nystagmus is the most sensitive and specific sign of vestibular lesions. The measurement of nystagmus pattern by infrared video goggle collected in clinic can provide a valuable diagnostic information for dizziness and vertigo. This paper mainly studies the automatic detection of torsional BPPV nystagmus based on deep learning, thus assisting clinicians diagnose dizziness and vertigo conveniently. In order to eliminate the invalid frames from the blinking of patients under observation, a convolutional neural network(ConvNet) based eye movement video condensation approach is proposed. When calibrating the moving pupil in the captured frame sequence, the Hough transform and trajectory tracking based on template matching are well combined to improve the robustness to eyelash occlusion and pupil deformation. In addition, the optical flow field of moving eyeball is exploited to characterize the torsion motion of torsional nystagmus, based on which a Torsion-aware Bi-Stream Identification Network (TBSIN) model is proposed. Furthermore, through label-error correction based on temporal consistency, we can merge multiple continuous torsional frames into torsional nystagmus segments for clinical diagnosis. Experiments are conducted on a clinically collected torsional nystagmus video dataset and promising experimental results show the effectiveness of the proposed approach. In particular, we achieve 85.73% and 81.00% in view of Accuracy and F1 measurements for frame-level identification, as well as IOU performance 67.45% for final torsional nystagmus segment localization.

The Role of Mobile Phone Camera Recordings in the Diagnosis of Meniere's Disease and Pathophysiological Implications.

This study aimed to understand if videos of the patients' nystagmus recorded by themselves during the attacks can help in the diagnosis of Meniere's disease (MD). Sixty patients (age range 32-78 years) who had vestibular attacks and hearing complaints admitted to Çukurova University Hospital Otolaryngology Department and a private office between September 2013 and January 2017 were included in this randomized clinical trial study. Two groups with 30 patients each were formed. The first group was asked to send eye-videos recorded during the attack, while the patients in the second group were followed with conventional methods. Twenty-six patients in the first group were able to send satisfactory eye movement videos; four patients were excluded due to repeated recording faults. Twenty-seven patients in the second group could be followed; three patients were lost to follow-up. The number of attacks and time needed to diagnose both groups were compared. The video group could be diagnosed in a shorter period compared to the control group. The diagnosis was made within two attacks (38 days) in the video group and within four attacks (92 days) in the control group. This study shows that cell phone camera recordings of nystagmus of the patients are very helpful to diagnose MD. These recordings can also be used as an adjunct to understand the pathophysiology of the disease.

Automatic Emotion Perception Using Eye Movement Information for E-Healthcare Systems.

Facing the adolescents and detecting their emotional state is vital for promoting rehabilitation therapy within an E-Healthcare system. Focusing on a novel approach for a sensor-based E-Healthcare system, we propose an eye movement information-based emotion perception algorithm by collecting and analyzing electrooculography (EOG) signals and eye movement video synchronously. Specifically, we extract the time-frequency eye movement features by firstly applying the short-time Fourier transform (STFT) to raw multi-channel EOG signals. Subsequently, in order to integrate time domain eye movement features (i.e., saccade duration, fixation duration, and pupil diameter), we investigate two feature fusion strategies: feature level fusion (FLF) and decision level fusion (DLF). Recognition experiments have been also performed according to three emotional states: positive, neutral, and negative. The average accuracies are 88.64% (the FLF method) and 88.35% (the DLF with maximal rule method), respectively. Experimental results reveal that eye movement information can effectively reflect the emotional state of the adolescences, which provides a promising tool to improve the performance of the E-Healthcare system.

Evaluation of the Effectiveness of a Gaze-Based Training Intervention on Latent Hazard Anticipation Skills for Young Drivers: A Driving Simulator Study

A PC-based training program (Road Awareness and Perception Training or RAPT; Pradhan et al., 2009), proven effective for improving young novice drivers’ hazard anticipation skills, did not fully maximize the hazard anticipation performance of young drivers despite the use of similar anticipation scenarios in both, the training and the evaluation drives. The current driving simulator experiment examined the additive effects of expert eye movement videos following RAPT training on young drivers’ hazard anticipation performance compared to video-only and RAPT-only conditions. The study employed a between-subject design in which 36 young participants (aged 18–21) were equally and randomly assigned to one of three experimental conditions, were outfitted with an eye tracker and drove four unique scenarios on a driving simulator to evaluate the effect of treatment on their anticipation skills. The results indicate that the young participants that viewed the videos of expert eye movements following the completion of RAPT showed significant improvements in their hazard anticipation ability (85%) on the subsequent experimental evaluation drives compared to those young drivers who were only exposed to either the RAPT training (61%) or the Video (43%). The results further imply that videos of expert eye movements shown immediately after RAPT training may improve the drivers’ anticipation skills by helping them map and integrate the spatial and tactical knowledge gained in a training program within dynamic driving environments involving latent hazards.

A new preprocessing parameter estimation based on geodesic active contour model for automatic vestibular neuritis diagnosis.

The diagnostic of the vestibular neuritis (VN) presents many difficulties to traditional assessment methods This paper deals with a fully automatic VN diagnostic system based on nystagmus parameter estimation using a pupil detection algorithm. A geodesic active contour model is implemented to find an accurate segmentation region of the pupil. Hence, the novelty of the proposed algorithm is to speed up the standard segmentation by using a specific mask located on the region of interest. This allows a drastically computing time reduction and a great performance and accuracy of the obtained results. After using this fast segmentation algorithm, the obtained estimated parameters are represented in temporal and frequency settings. A useful principal component analysis (PCA) selection procedure is then applied to obtain a reduced number of estimated parameters which are used to train a multi neural network (MNN). Experimental results on 90 eye movement videos show the effectiveness and the accuracy of the proposed estimation algorithm versus previous work.

A Psychomotor Test for Assessment of Visuomotor Coordination during Performance of a Monotonous Target-Tracking Activity

Visuomotor coordination is required for successful performance of activities. Different types of tasks, such as getting into cars or working in static conditions require various coordination patterns with different levels of interaction between eye and hand movements. A psychomotor test was developed for analysis of impairments to visuomotor coordination induced by decreases in the level of arousal. The test is as follows: a small round spot (14 mm in diameter) was used as the target spot, and this moved across a monitor screen at a constant slow rate (12 mm/sec) in a circular orbit of diameter 70 mm with a period of 20 sec. Subjects were told: “Keep the mouse cursor inside the target object. If the cursor comes out of the moving target, this will be counted as an error.” Subjects’ reactivity levels were tested by introducing occasional presentations of an additional moving target, which had to be caught with the mouse as quickly as possible, with a click of the mouse button. The trajectory of the mouse cursor was recorded with a time resolution of 120 Hz. Eye movements were recorded using a contactless eye movement video detection system (Eyegaze Development System, LC Technologies, USA), which is based on reflection of infrared light from the cornea and allows the coordinates of gaze direction to be measured with a time resolution of 120 Hz. The monotonous nature of the test decreased the level of arousal and induced a drowsy state, which led to the occurrence of errors 30–40 min from the start of the experiment. The dynamics of the physiological level of arousal were identified electroencephalographically. Analysis of the time characteristics of slow tracking and fast saccadic eye movements and of hand movements showed them to be highly sensitive to decreases in work capacity induced by decreases in the level of arousal. Further development of this approach will facilitate the development of a contactless method for the rapid diagnosis of critical levels of drowsiness and for identifying the professional characteristics of human operators requiring high levels of visuomotor coordination.