Multisensory Feedback Research Articles

Integrated Gait Triggered Mixed Reality and Neurophysiological Monitoring as a Framework for Next-Generation Ambulatory Stroke Rehabilitation.

Brain stroke affects millions of people in the world every year, with 50 to 60 percent of stroke survivors suffering from functional disabilities, for which early and sustained post-stroke rehabilitation is highly recommended. However, approximately one third of stroke patients do not receive early in hospital rehabilitation programs due to insufficient medical facilities or lack of motivation. Gait triggered mixed reality (GTMR) is a cognitive-motor dual task with multisensory feedback tailored for lower-limb post-stroke rehabilitation, which we propose as a potential method for addressing these rehabilitation challenges. Simultaneous gait and EEG data from nine stroke patients was recorded and analyzed to assess the applicability of GTMR to different stroke patients, determine any impacts of GTMR on patients, and better understand brain dynamics as stroke patients perform different rehabilitation tasks. Walking cadence improved significantly for stroke patients and lower-limb movement induced alpha band power suppression during GTMR tasks. The brain dynamics and gait performance across different severities of stroke motor deficits was also assessed; the intensity of walking induced event related desynchronization (ERD) was found to be related to motor deficits, as classified by Brunnstrom stage. In particular, stronger lower-limb movement induced ERD during GTMR rehabilitation tasks was found for patients with moderate motor deficits (Brunnstrom stage IV). This investigation demonstrates the efficacy of the GTMR paradigm for enhancing lower-limb rehabilitation, explores the neural activities of cognitive-motor tasks in different stages of stroke, and highlights the potential for joining enhanced rehabilitation and real-time neural monitoring for superior stroke rehabilitation.

A Survey of Mid-Air Ultrasound Haptics and Its Applications

Ultrasound haptics is a contactless haptic technology that enables novel mid-air interactions with rich multisensory feedback. This article surveys recent advances in ultrasound haptic technology. We discuss the fundamentals of this haptic technology, how a variety of perceptible sensations are rendered, and how it is currently being used to enable novel interaction techniques. We summarize its strengths, weaknesses, and potential applications across various domains. We conclude with our perspective on key directions for this promising haptic technology.

Embodiment in Active Sport Tourism: An Autophenomenography of the Tour de France Alpine “Cols”

Existing conceptualizations of active sport tourism lack an empirical foundation in explaining the processes of embodiment through which active sport tourists engage with destination space. Adopting an autophenomenographic perspective, this paper explores embodiment in the context of a cycling tourism experience encompassing six iconic mountain passes within the French Alps synonymous with the Tour de France. Qualitative data draw attention to kinaesthetic and visceral sensations arising through multisensory feedback, along with affective responses produced as the body traverses venerated sport landscapes. This research highlights mind–body processes that shape mobile, active sport tourism experiences and provides an empirical and conceptual foundation to inform future studies of embodiment in active sport tourism.

A INFLUÊNCIA DA REALIDADE VIRTUAL SOBRE A VELOCIDADE DA MARCHA E AVALIAÇÃO DA SATISFAÇÃO DE INDIVÍDUOS COM DOENÇA DE PARKINSON

Virtual Reality (VR) is a promising tool used in the rehabilitation of individuals with Parkinson's Disease(PD) as it promotes motor learning through multisensory feedback in real time. This is an experimental study with a single arm whose objective was to analyze the influence of VR on gait speed and satisfaction assessment of individuals with PD. In the initial assessment (IA1) a 10-Meter Walk Test (10MWT) was applied, followed by the protocol of 10 sessions with VR using Nintendo WiiTM console. In the final evaluation (FE2), the 10MWT was reapplied, followed by the Abbreviated Patients’ Satisfaction with Mental Health Services Scale (SATIS-BR). The 10MWT showed 0.91 ± 0.20 in IA1 and 1 ± 0.22 in FE2 with no significant difference (p= 0.187) while SATIS-BR scored 4.8 ± 0.22 out of 5. It is concluded that despite the non-significant result, the gait speed was maintained, exposing expressive values of the participants' satisfaction.

Improvement of dynamic postural stability by an exercise program

BackgroundCentral processing of multi-sensory feedback and motor commands responsible for force production are critical for postural control. An exercise program was developed to realign spinal curvature, but its effect on postural control is unknown. Research QuestionTo what extent would the exercise program influence on center of pressure (CoP) sway on stable and unstable surfaces? MethodsSubjects (n = 30) were randomly assigned into one of three groups: exercise on a cylinder-shaped tube (98-cm length, 15-cm diameter, n = 10), exercise on a flat surface (n = 10), and a control group that laid supine on a flat surface (n = 10). Standing posture of each subject was quantified using anterior-, posterior-, and lateral-view photography. Each subject’s CoP sway was measured while standing on a static and dynamic platform with eyes open and eyes closed. Subjects were instructed to stand still when the platform was held stationary (e.g., no tilt) during the static condition. During the dynamic condition the platform was allowed to tilt in response to changes of CoP and subjects were instructed to maintain the platform in a horizontal position. ResultsOnly when subjects performed the exercise program on the tube, the angles of neck flexion and pelvis tilt decreased, and CoP sway in the sagittal, but not frontal plane, decreased during the dynamic platform conditions with both eyes open and eyes closed (p < 0.05). SignificanceIt is speculated that performing the exercise program on the tube might enhance a) central processing of somatosensory and vestibular inputs, b) motor commands responsible for force production in postural control, and c) biomechanical advantage by the realigned posture. The exercise program can be used by a variety of populations as home-exercise to realign the neck and pelvic posture and improve dynamic postural stability.

Hands-Free Accessible Digital Musical Instruments: Conceptual Framework, Challenges, and Perspectives

Exponential increases of available computational resources, miniaturization, and sensors, are enabling the development of digital musical instruments that use non-conventional interaction paradigms and interfaces. This scenario opens up new opportunities and challenges in the creation of accessible instruments to include persons with disabilities into music practice. This work focuses in particular on instruments dedicated to people who can not use limbs, for whom the only means for musical expression are the voice and a small number of traditional instruments. First, a modular and adaptable conceptual framework is discussed for the design of accessible digital musical instruments targeted at performers with motor impairments. Physical interaction channels available from the neck upwards (head, mouth, eyes, brain) are analyzed in terms of potential and limitations for musical interaction. Second, a systematic survey of previously developed instruments is presented: each is analyzed in terms of design choices, physical interaction channels and related sensors, mapping strategies, performer interface and feedback. As a result of this survey, several open research directions are discussed, including the use of unconventional interaction channels, musical control mappings, multisensory feedback, design, evaluation, and adaptation.

An Evaluation of a Wearable Assistive Device for Augmenting Social Interactions

Gaze behaviors contain rich information regarding a person's emotions and engagements. Reciprocal eye contact can invoke feelings of liking between two strangers. But blind people cannot perceive and establish the eye contact with sighted counterparts, causing their feelings of social isolation and low confidence in conversations. Thus, our research purpose is to let blind people perceive and react gaze behaviors in social interactions. A Social Glasses system has been implemented iteratively to deliver the multisensory feedback channels of the “eye contact”, integrating both visual and tactile feedback. Specifically, the system consists of a Social Glasses device and a tactile wristband, which are worn by a blind person. The Social Glasses simulates the natural gaze for the blind person, aiming at establishing the “eye contact” between blind and sighted people. The tactile wristband enables the blind person to perceive the corresponding tactile feedback when an “eye contact” happens. To test the system, we conducted a user experiment with 40 participants, including 10 blind-sighted pairs (N = 20) and 10 blindfolded-sighted pairs (N = 20), to see how it could help increase the communication quality between blind and sighted people, as well as to suggest implications for its design. Our main findings demonstrated that both the simulated gaze and the tactile feedback were significantly effective to enhance the communication quality in blind-sighted conversations. Overall, we contribute (1) empirical research findings on how a Social Glasses system enhances the communication quality in blind-sighted conversations; and (2) design principles to inform future assistive wearable device for augmenting social interactions.

Performance on the mouse vibration actuating search task is compromised by sleep deprivation.

As we go about our daily routines we are continuously bombarded with environmental feedback that requires appraisal and response. Sleep loss can compromise the efficiency by which these cognitive processes function. Operationally, poor performance caused by insufficient sleep translates to increased health and safety risks in settings where attention and timely and/or accurate decisions to respond are critical (e.g., at work, on the road, etc.). Current rodent tasks that assess altered cognition after sleep deprivation (SD) do not accurately model the continuous multisensory feedback that informs goal-oriented behavior in humans. Herein, we describe the vibration actuating search task (VAST), which consists of a vibrating open field with pseudo-randomly selected entrance and target destination points. To successfully complete a trial, mice use feedback from rotary motor-induced floor vibrations to navigate from the entrance point to the target destination. Sets of 20 trials were conducted on 3 consecutive days, and before testing on the third day control mice were undisturbed while other mice were sleep deprived for 10 h. On the first 2 days mice learned the task with high success rates. Alternatively, VAST performance was compromised following SD as measured by increased failures in task completion, time to target, time spent immobile, and decreased speed as compared with undisturbed mice. The VAST enables the analysis of continuous feedback via multiple sensory modalities in mice and is applicable to a variety of operational settings.NEW & NOTEWORTHY The vibration actuating search task (VAST) is a novel performance assay that uses continuous auditory and haptic feedback to motivate and direct search behaviors in mice. The VAST is rapidly acquired by mice and performance is disrupted by sleep deprivation. The VAST has practical application in occupational settings. The cognitive aspects of the sensorimotor integration in the VAST may prove useful for rodent models of neurodegenerative disease.

Brain-computer-interface technology with multisensory feedback for controlled ideomotor training in the rehabilitation of stroke patients

Восстановление функции руки у пациентов после инсульта является приоритетным направлением в нейрореабилитации. Развитие технологии интерфейса мозг-компьютер-экзоскелет кисти (ИМКЭ) качественно улучшило реабилитацию в этом направлении. Целью данного исследования было изучить факторы, влияющие на двигательное восстановление верхней конечности у пациентов после инсульта на фоне применения технологии ИМКЭ. Исследовали 24 пациента (14 мужчин, 10 женщин) в возрасте от 51 до 62 лет с единичным очагом инсультной этиологии, супратенториальной локализации. В 11 (45,6%) случаях левополушарное поражение, в 13 (54,4%) случаях — правополушарное. Давность инсульта — 4,0 (3,0; 12,0) месяца. Медиана МоСА — 25,0 (23,0; 27,0). Курс занятий ИМКЭ включал 9,5 (8,0; 10,0) процедур. Выявлена значимая умеренная корреляция между успешностью моторного представления (MIQ-RS) и эффективностью взаимодействия пациентов с ИМКЭ кисти. Пациенты с высокими показателями MIQ-RS 47,5 (32,0; 54,0) достоверно лучше взаимодействовали с ИМКЭ 63,0 (54,0; 67,0), R = 0,67 (p &lt; 0,05). У пациентов с высокими показателями MIQ-RS динамика двигательного восстановления была более выраженной: медиана ΔFugl-Meyer Assessment (FMA) составила 14 (8,0; 16,0) баллов против группы пациентов с низкими показателями MIQ-RS, ΔFMA — 10 (6,0; 13,0), но при этом не достигла статистически значимого уровня. Таким образом, выявлена взаимосвязь между способностью пациентов к моторному представлению (MIQ-RS) и эффективностью взаимодействия пациентов с ИМКЭ. Для выявления влияния этих факторов на динамику двигательного восстановления руки, вероятно, требуется продолжить исследование с большей выборкой.

Brain-computer-interface technology with multisensory feedback for controlled ideomotor training in the rehabilitation of stroke patients

Motor recovery of the upper limb is a priority in the neurorehabilitation of stroke patients. Advances in the brain-computer interface (BCI) technology have significantly improved the quality of rehabilitation. The aim of this study was to explore the factors affecting the recovery of the upper limb in stroke patients undergoing BCI-based rehabilitation with the robotic hand. The study recruited 24 patients (14 men and 10 women) aged 51 to 62 years with a solitary supratentorial stroke lesion. The lesion was left-hemispheric in 11 (45.6%) patients and right-hemispheric in 13 (54.4%) patients. Time elapsed from stroke was 4.0 months (3.0; 12.0). The median MoCa score was 25.0 (23.0; 27.0). The rehabilitation course consisted of 9.5 sessions (8.0; 10.0). We established a significant moderate correlation between motor imagery performance (the MIQ-RS score) and the efficacy of patient-BCI interaction. Patients with high MIQ-RS scores (47.5 (32.0; 54.0) achieved a better control of the BCI-driven hand exoskeleton (63.0 (54.0; 67.0), R = 0.67; p &lt; 0.05). Recovery dynamics were more pronounced in patients with high MIQ-RS scores: the median score on the Fugl-Meyer Assessment scale was 14 (8.0; 16.0) points vs 10 (6.0; 13.0) points in patients with low MIQ-RS scores. However, the difference was not significant. Thus, we established a correlation between a patient’s ability for motor imagery (MIQ-RS) and the efficacy of patient-BCI interaction. A larger patient sample might be necessary to assess the effect of these factors on motor recovery dynamics.

Current State of User Experience Evaluation in Virtual Reality: A Systematic Review from an Ergonomic Perspective

Virtual reality (VR) is receiving enough attention to be regarded as a revival era and technologies related to the implementation of VR systems continue to evolve. VR systems are applied not only in entertainment but also in various fields such as medicine, rehabilitation, education, engineering, and military (Aïm, Lonjon, Hannouche, &amp; Nizard, 2016; Howard, 2017; Lele, 2013). In particular, low-cost and immersive VR systems are commercialized to the general public, accelerating the revival of VR (Wang &amp; Lindeman, 2015). In VR system, the research from the viewpoint of human–computer interaction and user experience (UX) is required to provide a high sense of immersion to the user. Therefore, the purpose of this study is to provide a structural methodology for classifying current VR researches and to review UX evaluation of VR systems systematically to identify research trends and to clarify future research directions. This study followed systematic review protocol of (PRISMA) (Liberati et al., 2009). To cover a broad spectrum of perspectives of engineering and medical fields, six web databases were selected: Scopus, Web of Science, Science direct, IEEE Xplore, EBSCO, and ProQuest. The main search keywords were virtual reality and user experience. These two words can be used in acronyms or other words. As a result, four and three words were chosen for virtual reality and user experience, respectively (‘virtual reality’, ‘virtual environment’, ‘VR,’ and ‘VE’ were chosen as keywords for virtual reality, ‘user experience’, ‘UX,’ and ‘human experience’). In addition, the journal articles in English were searched only. After the screening process was completed, final articles were selected based on the full-text. In this process, there were two essential selection conditions. The selected articles should use VR system and evaluate UX component. No restrictions other than these conditions were made. As a result, 78 articles were found to be consistent with the purpose of this study. As a result, there were two main points of discussion about UX studies in a VR system. The first is related to the implementation of equipment and technology including input devices, output devices, feedback forms, platforms, and applications. The other is related to research methods including user characteristics, interactions, and evaluation method. With respect to hand input devices, conventional input devices such as keyboards and game pads were used in many cases compared to trackable devices. However, as implementation techniques for natural interaction such as gesture recognition or real-time tracking of the body parts have been extensively developed, UX research needs to be conducted on VR systems that apply these techniques. In relation to feedback, stimuli other than visual stimuli were not frequently provided. Since providing multiple types of stimuli simultaneously may increase the user’s immersion and sense of reality, it is necessary to intensively study the effect of multi-sensory feedback in the future. In addition, there is a lack of academic research on CAVE and motion platforms. Though CAVE and motion platforms are difficult to set up for experimentation because they are expensive to build and require large space, there is a need to continually expand the UX research on this platform since the public will have more opportunities to access these platforms. Regarding research methods, most of the studies have focused on subjective measurements, quantitative research, laboratory experiments, and episode UX. To comprehensively understand the overall UX, it is necessary to conduct a qualitative study such as observation of subjects experiencing a VR system, think aloud, or deep interview with them, rather than evaluating UX only through a questionnaire. In addition, there was no case in which UX was evaluated in terms of momentary UX. However, there is a limit to evaluating the subjective measurement such as immersion, presence, and motion sickness during usage by directly asking the user, since the VR system provides an immersive environment to the user. Thus, behavioral characteristic or physiological signal of users can be used as one of the evaluation indicators of these measurements. Today, new VR systems are emerging and VR-related technologies are expected to evolve steadily. In this context, the findings can contribute to future research directions and provide insights into conducting UX evaluation in VR system.

Impact of Physiological Signals Acquisition in the Emotional Support Provided in Learning Scenarios.

Physiological sensors can be used to detect changes in the emotional state of users with affective computing. This has lately been applied in the educational domain, aimed to better support learners during the learning process. For this purpose, we have developed the AICARP (Ambient Intelligence Context-aware Affective Recommender Platform) infrastructure, which detects changes in the emotional state of the user and provides personalized multisensorial support to help manage the emotional state by taking advantage of ambient intelligence features. We have developed a third version of this infrastructure, AICARP.V3, which addresses several problems detected in the data acquisition stage of the second version, (i.e., intrusion of the pulse sensor, poor resolution and low signal to noise ratio in the galvanic skin response sensor and slow response time of the temperature sensor) and extends the capabilities to integrate new actuators. This improved incorporates a new acquisition platform (shield) called PhyAS (Physiological Acquisition Shield), which reduces the number of control units to only one, and supports both gathering physiological signals with better precision and delivering multisensory feedback with more flexibility, by means of new actuators that can be added/discarded on top of just that single shield. The improvements in the quality of the acquired signals allow better recognition of the emotional states. Thereof, AICARP.V3 gives a more accurate personalized emotional support to the user, based on a rule-based approach that triggers multisensorial feedback, if necessary. This represents progress in solving an open problem: develop systems that perform as effectively as a human expert in a complex task such as the recognition of emotional states.

Multisensory perceptual awareness: Categorical or graded?

Neural evidence suggests that mechanisms associated with conscious access (i.e., the ability to report on a conscious state) are “all-or-none”. Upon crossing some threshold, neural signals are globally broadcast throughout the brain and allow conscious reports. However, whether subjective experience (phenomenal consciousness) is categorical (i.e., transitioning abruptly from unconscious to conscious states) or graded (i.e., characterized by multiple intermediate states) remains an open question. To address this issue, we built a series of artificial neural networks containing distinct feedback connectivity from “multisensory” to “unisensory” cortices. In line with consciousness theories, we operationalized perceptual consciousness by the presence of feedback from higher-order nodes back to unisensory nodes which allow ‘neural ignition’ – a rapid, non-linear boost in response putatively leading to phenomenal consciousness. When simulating how these networks responded to unisensory and multisensory inputs, we found the fastest responses for multisensory presentations associated with multisensory feedback, and the slowest responses for multisensory presentations without feedback. Most interestingly, despite being built in line with “all-or-none” models of consciousness, multisensory stimuli associated with unisensory feedback (i.e., auditory or visual), and hence consistent with unisensory phenomenology according to theories of consciousness, generated intermediate reaction times. To extend these models to human perception and performance, we conducted extensive psychophysical testing in 29 subjects who each completed 10 h of a multisensory cue-congruency task. Consistent with the modeling results, we found that reaction times to multisensory cues reported as unisensory were intermediate between those of fully aware and fully unaware cues. These results support the existence of graded forms of phenomenological consciousness that can be instantiated by simple neural networks built in line with “all-or-none” models of consciousness.

Cognition in Sensorimotor Control: Interfacing With the Posterior Parietal Cortex.

Millions of people worldwide are afflicted with paralysis from a disruption of neural pathways between the brain and the muscles. Because their cortical architecture is often preserved, these patients are able to plan movements despite an inability to execute them. In such people, brain machine interfaces have great potential to restore lost function through neuroprosthetic devices, circumventing dysfunctional corticospinal circuitry. These devices have typically derived control signals from the motor cortex (M1) which provides information highly correlated with desired movement trajectories. However, sensorimotor control simultaneously engages multiple cognitive processes such as intent, state estimation, decision making, and the integration of multisensory feedback. As such, cortical association regions upstream of M1 such as the posterior parietal cortex (PPC) that are involved in higher order behaviors such as planning and learning, rather than in encoding movement itself, may enable enhanced, cognitive control of neuroprosthetics, termed cognitive neural prosthetics (CNPs). We illustrate in this review, through a small sampling, the cognitive functions encoded in the PPC and discuss their neural representation in the context of their relevance to motor neuroprosthetics. We aim to highlight through examples a role for cortical signals from the PPC in developing CNPs, and to inspire future avenues for exploration in their research and development.

Experimentally induced limb-disownership in mixed reality

The seemingly stable construct of our bodily self depends on the continued, successful integration of multisensory feedback about our body, rather than its purely physical composition. Accordingly, pathological disruption of such neural processing is linked to striking alterations of the bodily self, ranging from limb misidentification to disownership, and even the desire to amputate a healthy limb. While previous embodiment research has relied on experimental setups using supernumerary limbs in variants of the Rubber Hand Illusion, we here used Mixed Reality to directly manipulate the feeling of ownership for one's own, biological limb. Using a Head-Mounted Display, participants received visual feedback about their own arm, from an embodied first-person perspective. In a series of three studies, in independent cohorts, we altered embodiment by providing visuotactile feedback that could be synchronous (control condition) or asynchronous (400 ms delay, Real Hand Illusion). During the illusion, participants reported a significant decrease in ownership of their own limb, along with a lowered sense of agency. Supporting the right-parietal body network, we found an increased illusion strength for the left upper limb as well as a modulation of the feeling of ownership during anodal transcranial direct current stimulation. Extending previous research, these findings demonstrate that a controlled, visuotactile conflict about one's own limb can be used to directly and systematically modulate ownership – without a proxy. This not only corroborates the malleability of body representation but questions its permanence. These findings warrant further exploration of combined VR and neuromodulation therapies for disorders of the bodily self.

High unbound bilirubin for age: a neurotoxin with major effects on the developing brain.

Neonatal hyperbilirubinemia is one of the most frequent diagnoses made in neonates. A high level of unconjugated bilirubin that is unbound to albumin is neurotoxic when the level exceeds age-specific thresholds or at lower levels in neonates with neurotoxic risk factors. Lower range of unbound bilirubin results in apoptosis, while moderate-to-high levels result in neuronal necrosis. Basal ganglia and various brain stem nuclei are more susceptible to bilirubin toxicity. Proposed mechanisms of bilirubin-induced neurotoxicity include excessive release of glutamate, mitochondrial energy failure, release of proinflammatory cytokines, and increased intracellular calcium concentration. These mechanisms are similar to the events that occur following hypoxic-ischemic insult in neonates. Severe hyperbilirubinemia in term neonates has been shown to be associated with increased risk for autism spectrum disorders. The neuropathological finding of bilirubin-induced neurotoxicity also includes cerebellar injury with a decreased number of Purkinje cells, and disruption of multisensory feedback loop between cerebellum and cortical neurons which may explain the clinical characteristics of autism spectrum disorders. Severe hyperbilirubinemia occurs more frequently in infants from low- and middle-income countries (LMIC). Simple devices to measure bilirubin, and timely treatment are essential to reduce neurotoxicity, and improve outcomes for thousands of neonates around the world.

Robotic interfaces for cognitive psychology and embodiment research: A research roadmap.

Advanced human-machine interfaces render robotic devices applicable to study and enhance human cognition. This turns robots into formidable neuroscientific tools to study processes such as the adaptation between a human operator and the operated robotic device and how this adaptation modulates human embodiment and embodied cognition. We analyze bidirectional human-machine interface (bHMI) technologies for transparent information transfer between a human and a robot via efferent and afferent channels. Even if such interfaces have a tremendous positive impact on feedback loops and embodiment, advanced bHMIs face immense technological challenges. We critically discuss existing technical approaches, mainly focusing on haptics, and suggest extensions thereof, which include other aspects of touch. Moreover, we point out other potential constraints such as limited functionality, semi-autonomy, intent-detection, and feedback methods. From this, we develop a research roadmap to guide understanding and development of bidirectional human-machine interfaces that enable robotic experiments to empirically study the human mind and embodiment. We conclude the integration of dexterous control and multisensory feedback to be a promising roadmap towards future robotic interfaces, especially regarding applications in the cognitive sciences. This article is categorized under: Computer Science > Robotics Psychology > Motor Skill and Performance Neuroscience > Plasticity.

Electroencephalographic Correlates of Continuous Postural Tasks of Increasing Difficulty

Cortical involvement in postural control is well recognized, however the role of non-visual afferents remains unclear. Parietal cortical areas are strongly implicated in vestibulo-spatial functions, but topographical localization during balance tasks remains limited. Here, we use electroencephalography (EEG) during continuous balance tasks of increasing difficulty at single electrode positions. Twenty-four healthy, right-handed individuals performed four balance tasks of increasing difficulty (bipedal and unipedal) and a seated control condition with eyes closed. Subjective ratings of task difficulty were obtained. EEG was recorded from 32 electrodes; 5 overlying sensory and motor regions of interest (ROIs) were chosen for further investigation: C3, Cz, C4, P3, P4. Spectral power and coherence during balance tasks were analyzed in theta (4–8 Hz) and alpha (8–12 Hz) bands. Alpha power reduced as task difficulty increased and this reduction correlated with subjective difficulty ratings. Alpha coherence increased with task difficulty between C3–Cz–C4 electrode pairs. Differential changes in power were observed in Cz, suggestive of a distinct role at this electrode location, which captures lower limb cortical representation. Hemispheric asymmetry was observed, as reflected by greater reductions in theta and alpha power in right-sided areas. Our results demonstrate the functional importance of bilateral central and parietal cortices in continuous balance control. The hemispheric asymmetry observed implies that the non-dominant hemisphere is involved with online monitoring of postural control. Although the posterior parietal asymmetry found may relate to vestibular, somatosensory or multisensory feedback processing, we argue that the finding relates to active balance control rather than simple sensory-intake or reflex circuit activation.

Distortion of Visuo-Motor Temporal Integration in Apraxia: Evidence From Delayed Visual Feedback Detection Tasks and Voxel-Based Lesion-Symptom Mapping.

Limb apraxia is a higher brain dysfunction that typically occurs after left hemispheric stroke and its cause cannot be explained by sensory disturbance or motor paralysis. The comparison of motor signals and visual feedback to generate errors, i.e., visuo-motor integration, is important in motor control and motor learning, which may be impaired in apraxia. However, in apraxia after stroke, it is unknown whether there is a specific deficit in visuo-motor temporal integration compared to visuo-tactile and visuo-proprioceptive temporal integration. We examined the precision of visuo-motor temporal integration and sensory-sensory (visuo-tactile and visuo-proprioception) temporal integration in apraxia after stroke by using a delayed visual feedback detection task with three different conditions (tactile, passive movement, and active movement). The delay detection threshold and the probability curve for delay detection obtained in this task were quantitative indicators of the respective temporal integration functions. In addition, we performed subtraction and voxel-based lesion-symptom mapping to identify the brain lesions responsible for apraxia and deficits in visuo-motor temporal integration. The behavioral experiments showed that the delay detection threshold was extended and that the probability curve for delay detection was less steep in apraxic patients compared to controls (pseudo-apraxic patients and unaffected patients), only for the active movement condition, and not for the tactile and passive movement conditions. Furthermore, the severity of apraxia was significantly correlated with the delay detection threshold and the steepness of the probability curve in the active movement condition. These results indicated that multisensory (i.e., visual, tactile, and proprioception) feedback was normally temporally integrated, but motor prediction and visual feedback were not correctly temporally integrated in apraxic patients. That is, apraxic patients had difficulties with visuo-motor temporal integration. Lesion analyses revealed that both apraxia and the distortion of visuo-motor temporal integration were associated with lesions in the fronto-parietal motor network, including the left inferior parietal lobule and left inferior frontal gyrus. We suppose that damage to the left inferior fronto-parietal network could cause deficits in motor prediction for visuo-motor temporal integration, but not for sensory-sensory (visuo-tactile and visuo-proprioception) temporal integration, leading to the distortion of visuo-motor temporal integration in patients with apraxia.

On-Road Evaluation of In-vehicle Interface Characteristics and Their Effects on Performance of Visual Detection on the Road and Manual Entry.

This study investigated the impact of in-vehicle interface characteristics on drivers' multitasking performance measures relating to visual attention management, concerning the distraction potential of in-vehicle touchscreens. Compared with physical controls providing drivers with naturalistic nonvisual cues, in-vehicle touchscreen interaction relies on vision to a greater extent, leading to more time with eyes off the road and concerns for safety. Little is known from existing research about the extent to which synthetic feedback of in-vehicle touchscreens support visual attention of multitasking drivers, while automakers are increasingly incorporating nondriving functions into in-vehicle touchscreens. Twenty-nine participants drove an instrumented vehicle on a closed course and acknowledged visual probes obscured on the roadside, while performing a manual data entry task with input interfaces mounted on the center console. The interfaces differed by interface type, key feedback modality, and key size; the configuration of interface characteristics was the within-subject variable. The collected data include performance measures concerning visual detection and touchscreen interaction, in addition to perceived workload. The addition of nonvisual feedback to touchscreen interaction significantly improved accuracy and promptness of visual detection. No significant difference was found between different sizes of touchscreen keys when synthetic nonvisual feedback was available. Given multisensory feedback, no measure showed a difference between touchscreen conditions and a physical keypad. The provision of synthetic nonvisual feedback to touchscreen interaction can support visual attention and enhance multitasking performance in driving. This study can inform in-vehicle interface designers and policy makers concerned with distracted driving and safety.