Visual Motion Research Articles

The application of immersive multimedia information technology in the teaching of vocal music

Abstract In this paper, immersive multimedia information technology is utilized to perform frame-splitting and window-adding operations on vocal signals in college vocal music teaching to facilitate the extraction of vocal signal features. According to the Principal Component Analysis feature space projection and Relief feature selection, the vocal emotion regression model combining multimedia information technology is constructed, and the structure of the vocal emotion regression model and its teaching application are explored in the vocal emotion regression model. Determine the research object and method, according to the research program on the integration of multimedia information technology of vocal music teaching research design and implementation, and the use of statistical analysis of multimedia information technology-based vocal music teaching empirical analysis. The results show that the subjects had the best emotional experience with an emotional immersion degree of 5.651 when the music of the calm category and the visual music motion picture of the calm category constituted the visual music and the vocal music immersion based on the vocal music emotional regression model of “combining multimedia information technology” was significantly enhanced when the emotional type of the music and the emotional type of the dynamic picture of the visual music was the same. This study carries out college vocal music teaching based on students’ emotional characteristics, which is of great significance to improve the quality of college vocal music teaching and students’ musical literacy.

Visual Cue Effects on a Classification Accuracy Estimation Task in Immersive Scatterplots.

Immersive visualization in virtual reality (VR) allows us to exploit visual cues for perception in 3D space, yet few existing studies have measured the effects of visual cues. Across a desktop monitor and a head-mounted display (HMD), we assessed scatterplot designs which vary their use of visual cues-motion, shading, perspective (graphical projection), and dimensionality-on two sets of data. We conducted a user study with a summary task in which 32 participants estimated the classification accuracy of an artificial neural network from the scatterplots. With Bayesian multilevel modeling, we capture the intricate visual effects and find that no cue alone explains all the variance in estimation error. Visual motion cues generally reduce participants' estimation error; besides this motion, using other cues may increase participants' estimation error. Using an HMD, adding visual motion cues, providing a third data dimension, or showing a more complicated dataset leads to longer response times. We speculate that most visual cues may not strongly affect perception in immersive analytics unless they change people's mental model about data. In summary, by studying participants as they interpret the output from a complicated machine learning model, we advance our understanding of how to use the visual cues in immersive analytics.

Reorganization of functional brain networks in healthy aging and mild cognitive impairment during the visual motion discrimination task

AbstractBackgroundPreviously, we reported that there was significant decrease in spectral power for eyes‐closed resting state EEG (rsEEG) immediately following a visual motion discrimination task as compared to before task engagement in older adults – the so‐called Task Aftereffect (TA). Further, we have also shown that rsEEG TA differentiated between older adults with mild cognitive impairment (MCI) and healthy controls (HC), with the MCI participants showing more integrated topology in post‐ as compared to pre‐cognitive task rsEEG. We now report EEG topological changes during task performance with regards to: changes in functional network organization from the beginning to the end of the task and evidence for significant differences in topology between MCI patients and HC.MethodWe studied a sample of community dwelling African American older participants over age 60 years, with consensus‐based diagnosis for 51 HC and 39 MCI. EEG epochs extracted for analysis were two‐second epochs before the motion onset of each stimulus at the beginning, middle, and end of the full task. Functional connectivity between channels was assessed by the phase lag index. Minimum spanning tree (MST) was used to characterize network topology.ResultTime effect was confirmed, as network organization in all participants significantly improved with task progression. On average, brain topology became more integrated with higher leaf fraction and smaller average eccentricity. Moreover, our results show differences between groups in MST parameters over the whole task. Most differences were observed at the earliest timepoint at lower frequency range (theta and lower alpha) where average eccentricity was significantly decreased. Leaf fraction and maximal betweenness centrality were significantly increased in the MCI group as compared to HC.ConclusionThe change in network organization suggests that MCI require greater network integration across brain areas before repeated motion onset presentations. This could reflect compensatory brain mechanisms, especially as both groups did not show significant differences in task performance. These findings enhance our understanding of the functional brain network organization immediately prior to repeated responses in a visual perceptual task in MCI compared to HC. This emphasizes the potential for using graph‐theoretic measures as a valid tool for the early detection of cognitive impairment.

Vibration analysis of drum membrane using computer vision

The real time experience of what one sees and what one hears is enriching but if one can visualize what is audible will be fascinating. We suggest a simpler approach for sound localization that directly collects motion signals based on visual movements. In this study, frequency has been detected at various location of drum using computer vision method. A smartphone camera set up on a tripod next to the drum has captured these vibrations at approximately 240 frames per second. To determine the frequency, a Fourier analysis was performed on the data collected from the frames. We detected frequency of ~94 Hz at multiple spots by analyzing the motions of a point or region of interest (ROI) at membrane of drum. The frequency of drum membrane detected by a mobile microphone was found to be equivalent to the frequency observed using the proposed computer-vision method. Similar concepts can be employed in understanding the fundamentals of sounds generated by various musical instruments.

Different spectral sensitivities of ON- and OFF-motion pathways enhance the detection of approaching color objects in Drosophila

Color and motion are used by many species to identify salient objects. They are processed largely independently, but color contributes to motion processing in humans, for example, enabling moving colored objects to be detected when their luminance matches the background. Here, we demonstrate an unexpected, additional contribution of color to motion vision in Drosophila. We show that behavioral ON-motion responses are more sensitive to UV than for OFF-motion, and we identify cellular pathways connecting UV-sensitive R7 photoreceptors to ON and OFF-motion-sensitive T4 and T5 cells, using neurogenetics and calcium imaging. Remarkably, this contribution of color circuitry to motion vision enhances the detection of approaching UV discs, but not green discs with the same chromatic contrast, and we show how this could generalize for systems with ON- and OFF-motion pathways. Our results provide a computational and circuit basis for how color enhances motion vision to favor the detection of saliently colored objects.

Examining the Association Between the Computer-Aided Scoring and Manual Scoring for the Beery Buktenica Developmental Test of Visual Motor Integration (Beery VMI) and Children’s Handwriting Skills: A Convergent Validity Study

ABSTRACT Introduction Visual motor integration (VMI) is an important underlying mechanism for children’s handwriting. The Beery-Buktenica Developmental Test of Visual-Motor Integration (Beery VMI) is commonly used to assess children’s VMI skills, using a manual scoring method which can be prone to individual subjectivity and impact its sensitivity. This study investigated the use of the Computer-Aided Scoring System for scoring the Berry VMI (CASS-Beery VMI) to assess school-aged children’s VMI and handwriting skills. Specifically, the convergent validity between the CASS-Beery VMI and three standardized handwriting assessments was investigated. Methods A quantitative cross-sectional design was used. Thirty-five neurotypical Australian students aged 6–9 years completed the Beery-VMI–6th edition, Minnesota Handwriting Assessment (MHA), Evaluation Tool of Children’s Handwriting – Manuscript (ETCH-M) and the Here’s How I Write Assessment – Child version (HHIW-C). The Berry-VMI was scored using its original scoring method and the CASS-Beery VMI. Thirty parents of the child participants completed the Here’s How I Write Assessment – Parent version (HHIW-P). Descriptive statistics, Spearman rho correlations and regression analyses using bootstrapping were completed. Results The CASS-Beery VMI error scores were significantly correlated with the Beery-VMI score, and subscales of the MHA, ETCH and HHIW-P scores (r = 0.341– −0.555; p = .045–0.001). The CASS-Beery VMI error scores and Beery VMI raw scores were found to be significant predictors of the ETCH-M number legibility scores (adjusted R2 = 0.209; p = 0.017) and MHA form scores (adjusted R2 = 0.220; p = 0.007). Individually, the CASS-Beery VMI score was a significant predictor of the ETCH-M word legibility scores (Adjusted R2 = 0.305; p = 0.001). Conclusion Findings indicate that the CASS-Beery VMI can be used to assess children’s VMI skills. The significant correlations between the CASS-Beery VMI and other standardized handwriting assessments (MHA, ETCH-M and HHIW) adds to the body of convergent validity evidence that the CASS-Beery VMI can be a sensitive scoring method for Beery VMI when assessing children’s handwriting performance.

Neck stabilization through sensory integration of vestibular and visual motion cues.

To counteract gravity, trunk motion, and other perturbations, the human head-neck system requires continuous muscular stabilization. In this study, we combine a musculoskeletal neck model with models of sensory integration (SI) to unravel the role of vestibular, visual, and muscle sensory cues in head-neck stabilization and relate SI conflicts and postural instability to motion sickness. A 3D multisegment neck model with 258 Hill-type muscle elements was extended with postural stabilization using SI of vestibular (semicircular and otolith) and visual (rotation rate, verticality, and yaw) cues using the multisensory observer model (MSOM) and the subjective vertical conflict model (SVC). Dynamic head-neck stabilization was studied using empirical datasets, including 6D trunk perturbations and a 4 m/s2 slalom drive inducing motion sickness. Recorded head translation and rotation are well matched when using all feedback loops with MSOM or SVC or assuming perfect perception. A basic version of the model, including muscle, but omitting vestibular and visual perception, shows that muscular feedback can stabilize the neck in all conditions. However, this model predicts excessive head rotations in conditions with trunk rotation and in the slalom. Adding feedback of head rotational velocity sensed by the semicircular canals effectively reduces head rotations at mid-frequencies. Realistic head rotations at low frequencies are obtained by adding vestibular and visual feedback of head rotation based on the MSOM or SVC model or assuming perfect perception. The MSOM with full vision well captures all conditions, whereas the MSOM excluding vision well captures all conditions without vision. The SVC provides two estimates of verticality, with a vestibular estimate SVCvest, which is highly effective in controlling head verticality, and an integrated vestibular/visual estimate SVCint which can complement SVCvest in conditions with vision. As expected, in the sickening drive, SI models imprecisely estimate verticality, resulting in sensory conflict and postural instability. The results support the validity of SI models in postural stabilization, where both MSOM and SVC provide credible results. The results in the sickening drive show imprecise sensory integration to enlarge head motion. This uniquely links the sensory conflict theory and the postural instability theory in motion sickness causation.

Visual objects approaching the body modulate subsequent somatosensory processing at 4 months of age

We asked whether, in the first year of life, the infant brain can support the dynamic crossmodal interactions between vision and somatosensation that are required to represent peripersonal space. Infants aged 4 (n = 20, 9 female) and 8 (n = 20, 10 female) months were presented with a visual object that moved towards their body or receded away from it. This was presented in the bottom half of the screen and not fixated upon by the infants, who were instead focusing on an attention getter at the top of the screen. The visual moving object then disappeared and was followed by a vibrotactile stimulus occurring later in time and in a different location in space (on their hands). The 4-month-olds’ somatosensory evoked potentials (SEPs) were enhanced when tactile stimuli were preceded by unattended approaching visual motion, demonstrating that the dynamic visual-somatosensory cortical interactions underpinning representations of the body and peripersonal space begin early in the first year of life. Within the 8-month-olds’ sample, SEPs were increasingly enhanced by (unexpected) tactile stimuli following receding visual motion as age in days increased, demonstrating changes in the neural underpinnings of the representations of peripersonal space across the first year of life.

Data-Driven Advancements in Lip Motion Analysis: A Review

This work reviews the dataset-driven advancements that have occurred in the area of lip motion analysis, particularly visual lip-reading and visual lip motion authentication, in the deep learning era. We provide an analysis of datasets and their usage, creation, and associated challenges. Future research can utilize this work as a guide for selecting appropriate datasets and as a source of insights for creating new and innovative datasets. Large and varied datasets are vital to a successful deep learning system. There have been many incredible advancements made in these fields due to larger datasets. There are indications that even larger, more varied datasets would result in further improvement upon existing systems. We highlight the datasets that brought about the progression in lip-reading systems from digit- to word-level lip-reading, and then from word- to sentence-level lip-reading. Through an in-depth analysis of lip-reading system results, we show that datasets with large amounts of diversity increase results immensely. We then discuss the next step for lip-reading systems to move from sentence- to dialogue-level lip-reading and emphasize that new datasets are required to make this transition possible. We then explore lip motion authentication datasets. While lip motion authentication has been well researched, it is not very unified on a particular implementation, and there is no benchmark dataset to compare the various methods. As was seen in the lip-reading analysis, large, diverse datasets are required to evaluate the robustness and accuracy of new methods attempted by researchers. These large datasets have pushed the work in the visual lip-reading realm. Due to the lack of large, diverse, and publicly accessible datasets, visual lip motion authentication research has struggled to validate results and real-world applications. A new benchmark dataset is required to unify the studies in this area such that they can be compared to previous methods as well as validate new methods more effectively.

Interception of vertically approaching objects: temporal recruitment of the internal model of gravity and contribution of optical information.

Introduction: Recent views posit that precise control of the interceptive timing can be achieved by combining on-line processing of visual information with predictions based on prior experience. Indeed, for interception of free-falling objects under gravity's effects, experimental evidence shows that time-to-contact predictions can be derived from an internal gravity representation in the vestibular cortex. However, whether the internal gravity model is fully engaged at the target motion outset or reinforced by visual motion processing at later stages of motion is not yet clear. Moreover, there is no conclusive evidence about the relative contribution of internalized gravity and optical information in determining the time-to-contact estimates. Methods: We sought to gain insight on this issue by asking 32 participants to intercept free falling objects approaching directly from above in virtual reality. Object motion had durations comprised between 800 and 1100ms and it could be either congruent with gravity (1g accelerated motion) or not (constant velocity or -1g decelerated motion). We analyzed accuracy and precision of the interceptive responses, and fitted them to Bayesian regression models, which included predictors related to the recruitment of a priori gravity information at different times during the target motion, as well as based on available optical information. Results: Consistent with the use of internalized gravity information, interception accuracy and precision were significantly higher with 1g motion. Moreover, Bayesian regression indicated that interceptive responses were predicted very closely by assuming engagement of the gravity prior 450ms after the motion onset, and that adding a predictor related to on-line processing of optical information improved only slightly the model predictive power. Discussion: Thus, engagement of a priori gravity information depended critically on the processing of the first 450 ms of visual motion information, exerting a predominant influence on the interceptive timing, compared to continuously available optical information. Finally, these results may support a parallel processing scheme for the control of interceptive timing.

The effects of synchronous and asynchronous steady-state auditory-visual motion on EEG characteristics in healthy young adults

The brain’s integration of motion information from visual and auditory senses is essential for human to be able to duly respond to the dynamic living environment. Research on the impact of steady-state auditory motion and steady-state visual motion synchronous stimuli and minor asynchronous stimuli on electroencephalogram (EEG) is lacking. This study designed steady-state visual motion paradigms, steady-state auditory motion paradigms, steady-state auditory-visual synchronous motion paradigms, and steady-state auditory-visual asynchronous motion paradigms. Then the effects of steady-state auditory-visual motion stimuli on EEG were analyzed from the aspects of time domain, frequency domain, connectivity, and source localization. Our study showed that the synchronous steady-state auditory-visual motion stimuli and the asynchronous steady-state auditory-visual motion stimuli can enhance brain responses in a subadditive mode and significantly activated auditory and visual integration areas compared to single-modal visual and auditory stimuli. Moreover, asynchronous steady-state auditory-visual motion stimuli activated the Anterior Cingulate (ACG) compared to synchronous steady-state auditory-visual motion stimuli. Our study demonstrated that ACG regions were involved in conflicting processing of steady-state auditory-visual motion information. This study further enriches the cutting edge of auditory-visual motion integration.

Visual–Motion–Interaction-Guided Pedestrian Intention Prediction Framework

Visual–Motion–Interaction-Guided Pedestrian Intention Prediction Framework

Visual Motor Assessment of Children with Autism Spectrum Disorder: Comparing Performance and Considerations for Assessment

ABSTRACT Standardized assessment of children with autism spectrum disorder (ASD) can produce inaccurate results due to behaviors exhibited during assessment. This study developed the Occupational Therapy Observation Tool—Adjustment Support Details for children with autism (OTOT-ASD) to investigate the behaviors demonstrated, and the reasonable supports required, during administration of formal visual motor assessments. The OTOT-ASD was used to record student’s behaviors and accommodations necessary to support their assessment when completing the Beery Buktenica Developmental Test of Visual Motor Integration 6th Ed. (Beery VMI) and a new assessment, the Hartley Knows Writing Shapes Assessment Version 3 (HKWSA-V3). This study evaluated the psychometric properties of the HKWSA-V3 in the assessment of 15 children with ASD. Preliminary findings were promising, although further investigation with a larger sample is required. The OTOT-ASD enabled high incidences of behaviors disruptive to testing and supportive accommodations to be captured during assessment. As a new tool, the OTOT-ASD may improve the clinical utility of formal visual motor assessments via enabling the administering occupational therapist to accurately make and record reasonable adjustments to support the child’s true performance capacity. Further development of the OTOT-ASD is recommended to evaluate the generalizability of the tool for use with children with ASD.

Comparative neuroanatomy of the central nervous system in web-building and cursorial hunting spiders.

Spiders (Araneae) include cursorial species that stalk their prey and more stationary species that use webs for prey capture. While many cursorial hunting spiders rely on visual cues, web-building spiders use vibratory cues (mechanosensation) for prey capture. We predicted that the differences in primary sensory input between the species are mirrored by differences in the morphology/architecture of the central nervous system (CNS). Here, we investigated the CNS anatomy of four spider species, two cursorial hunters Pardosa amentata (Lycosidae) and Marpissa muscosa (Salticidae), and two web-building hunters Argiope bruennichi (Araneidae) and Parasteatoda tepidariorum (Theridiidae). Their CNS was analyzed using Bodian silver impregnations, immunohistochemistry, and microCT analysis. We found that there are major differences between species in the secondary eye pathway of the brain that pertain to first-order, second-order, and higher order brain centers (mushroom bodies [MB]). While P. amentata and M. muscosa have prominent visual neuropils and MB, these are much reduced in the two web-building species. Argiope bruennichi lacks second-order visual neuropils but has specialized photoreceptors that project into two distinct visual neuropils, and P. tepidariorum lacks MB, suggesting that motion vision might be absent in this species. Interestingly, the differences in the ventral nerve cord are much less pronounced, but the web-building spiders have proportionally larger leg neuropils than the cursorial spiders. Our findings suggest that the importance of visual information is much reduced in web-building spiders, compared to cursorial spiders, while processing of mechanosensory information requires the same major circuits in both web-building and cursorial hunting spiders.

Truly pattern: Nonlinear integration of motion signals is required to account for the responses of pattern cells in rat visual cortex.

A key feature of advanced motion processing in the primate dorsal stream is the existence of pattern cells-specialized cortical neurons that integrate local motion signals into pattern-invariant representations of global direction. Pattern cells have also been reported in rodent visual cortex, but it is unknown whether the tuning of these neurons results from truly integrative, nonlinear mechanisms or trivially arises from linear receptive fields (RFs) with a peculiar geometry. Here, we show that pattern cells in rat primary (V1) and lateromedial (LM) visual cortex process motion direction in a way that cannot be explained by the linear spatiotemporal structure of their RFs. Instead, their tuning properties are consistent with and well explained by those of units in a state-of-the-art neural network model of the dorsal stream. This suggests that similar cortical processes underlay motion representation in primates and rodents. The latter could thus serve as powerful model systems to unravel the underlying circuit-level mechanisms.

Visual motion detection thresholds can be reliably measured during walking and standing.

In upright standing and walking, the motion of the body relative to the environment is estimated from a combination of visual, vestibular, and somatosensory cues. Associations between vestibular or somatosensory impairments and balance problems are well established, but less is known whether visual motion detection thresholds affect upright balance control. Typically, visual motion threshold values are measured while sitting, with the head fixated to eliminate self-motion. In this study we investigated whether visual motion detection thresholds: (1) can be reliably measured during standing and walking in the presence of natural self-motion; and (2) differ during standing and walking. Twenty-nine subjects stood on and walked on a self-paced, instrumented treadmill inside a virtual visual environment projected on a large dome. Participants performed a two-alternative forced choice experiment in which they discriminated between a counterclockwise ("left") and clockwise ("right") rotation of a visual scene. A 6-down 1-up adaptive staircase algorithm was implemented to change the amplitude of the rotation. A psychometric fit to the participants' binary responses provided an estimate for the detection threshold. We found strong correlations between the repeated measurements in both the walking (R = 0.84, p < 0.001) and the standing condition (R = 0.73, p < 0.001) as well as good agreement between the repeated measures with Bland-Altman plots. Average thresholds during walking (mean = 1.04°, SD = 0.43°) were significantly higher than during standing (mean = 0.73°, SD = 0.47°). Visual motion detection thresholds can be reliably measured during both walking and standing, and thresholds are higher during walking.

Differential Developmental Neurotoxicity and Tissue Uptake of the Per- and Polyfluoroalkyl Substance Alternatives, GenX and PFBS.

Per- and polyfluoroalkyl substances (PFAS) are a class of synthetic chemicals with several applications. Multiple adverse health effects are reported for longer carbon chain (≤C8) PFAS. Shorter carbon chain PFAS, [e.g., hexafluoropropylene oxide dimer acid (HFPO-DA; GenX) and perfluorobutanesulfonic acid (PFBS)] were introduced as alternatives. Past studies indicate that longer-chain PFAS are neurotoxic targeting the dopamine pathway, but it is not known if shorter-chain PFAS act similarly. This study aimed to evaluate developmental neurotoxicity and tissue uptake of GenX and PFBS using the zebrafish (Danio rerio). First, acute toxicity was assessed by measuring LC50 at 120 h postfertilization (hpf). Body burden was determined after embryonic exposure (1-72 hpf) to sublethal concentrations of GenX or PFBS by LC-ESI-MS/MS. Locomotor activity using a visual motor response assay at 120 hpf and dopamine levels at 72 hpf was assessed after embryonic exposure. PFBS was more acutely toxic and bioaccumulative than GenX. GenX and PFBS caused hyperactivity at 120 hpf, but stronger behavioral alterations were observed for PFBS. An increase in whole organism dopamine occurred at 40 ppb of GenX, while a decrease was observed at 400 ppb of PFBS. Differences detected in dopamine for these two PFAS indicate differential mechanisms of developmental neurotoxicity.

Tug-of-Peace: Visual Rivalry and Atypical Visual Motion Processing in MECP2 Duplication Syndrome of Autism.

Extracting common patterns of neural circuit computations in the autism spectrum and confirming them as a cause of specific core traits of autism is the first step toward identifying cell-level and circuit-level targets for effective clinical intervention. Studies in humans with autism have identified functional links and common anatomic substrates between core restricted behavioral repertoire, cognitive rigidity, and overstability of visual percepts during visual rivalry. To study these processes with single-cell precision and comprehensive neuronal population coverage, we developed the visual bistable perception paradigm for mice based on ambiguous moving plaid patterns consisting of two transparent gratings drifting at an angle of 120°. This results in spontaneous reversals of the perception between local component motion (plaid perceived as two separate moving grating components) and integrated global pattern motion (plaid perceived as a fused moving texture). This robust paradigm does not depend on the explicit report of the mouse, since the direction of the optokinetic nystagmus (OKN) is used to infer the dominant percept. Using this paradigm, we found that the rate of perceptual reversals between global and local motion interpretations is reduced in the methyl-CpG-binding protein 2 duplication syndrome (MECP2-ds) mouse model of autism. Moreover, the stability of local motion percepts is greatly increased in MECP2-ds mice at the expense of global motion percepts. Thus, our model reproduces a subclass of the core features in human autism (reduced rate of visual rivalry and atypical perception of visual motion). This further offers a well-controlled approach for dissecting neuronal circuits underlying these core features.

EARLY-ONSET OF FAMILIAL EXUDATIVE VITREORETINOPATHY: Clinical Characteristics, Management, and Outcomes.

To describe the clinical characteristics, management, and outcomes of toddlers (under the age of 3) diagnosed with familial exudative vitreoretinopathy. In this retrospective study, patients diagnosed with familial exudative vitreoretinopathy before the age of 3 were included. Presenting characteristics, genetic testing, management, and outcomes were collected. A total of 54 patients (108 eyes) with a mean age at diagnosis of 10.9 ± 2.6 months were included. Poor visual behavior (33%) and strabismus (26%) were the most common presenting symptoms, whereas screening only represented 11%. About half of included patients had a severe disease (stages 4 and 5). Genetic testing was positive in 40.7% of patients with 24% having a family history of familial exudative vitreoretinopathy. LRP5 was the most prevalent mutation (54.5%).Surgery was performed in 44.4% of eyes and was successful in 69.8% of cases. Failure exclusively occurred in eyes with severe stages. Among eyes evaluated for visual acuity (72 eyes), most (76.4%) had a vision of hand motion or better. Familial exudative vitreoretinopathy tended to be worse with earlier age at diagnosis, subsequently affecting the prognosis. Surgical intervention was common and primarily included lens-sparing vitrectomy and combined lensectomy and vitrectomy. Surgical success hinged on the stage of the disease.

Comparison of uni- and multimodal motion stimulation on visual neglect: A proof-of-concept study

Spatial neglect is characterized by the failure to attend stimuli presented in the contralesional space. Typically, the visual modality is more severely impaired than the auditory one. This dissociation offers the possibility of cross-modal interactions, whereby auditory stimuli may have beneficial effects on the visual modality. A new auditory motion stimulation method with music dynamically moving from the right to the left hemispace has recently been shown to improve visual neglect. The aim of the present study was twofold: a) to compare the effects of unimodal auditory against visual motion stimulation, i.e., smooth pursuit training, which is an established therapeutical approach in neglect therapy and b) to explore whether a combination of auditory + visual motion stimulation, i.e., multimodal motion stimulation, would be more effective than unimodal auditory or visual motion stimulation. 28 patients with left-sided neglect due to a first-ever, right-hemispheric subacute stroke were included. Patients either received auditory, visual, or multimodal motion stimulation. The between-group effect of each motion stimulation condition as well as a control group without motion stimulation was investigated by means of a one-way ANOVA with the patient’s visual exploration behaviour as an outcome variable. Our results showed that unimodal auditory motion stimulation is equally effective as unimodal visual motion stimulation: both interventions significantly improved neglect compared to the control group. Multimodal motion stimulation also significantly improved neglect, however, did not show greater improvement than unimodal auditory or visual motion stimulation alone. Besides the established visual motion stimulation, this proof-of-concept study suggests that auditory motion stimulation seems to be an alternative promising therapeutic approach to improve visual attention in neglect patients. Multimodal motion stimulation does not lead to any additional therapeutic gain. In neurorehabilitation, the implementation of either auditory or visual motion stimulation seems therefore reasonable.