Motion Feedback Research Articles

THE USE OF A COMPLEX “BRAIN-COMPUTER INTERFACE AND EXO-SKELETON” AND MOVEMENT IMAGINATION TECHNIQUE FOR POST-STROKE REHABILITATION

Background: Efficacy of physical exercise and movement imagination for restoration of motor dysfunction after a stroke is seen as proven. However, the use of movement imagination is complicated by impossibility of objective and subjective control over the exercise, as well as by the absence of their motor support. The brain-computer interface based on electroencephalography is a technique that enables a feedback during movement imagination. Materials and methods: We assessed 10 patients (6 men and 4 women) aged from 30 to 66 years (mean age, 47 ± 7.7 years) with an ischemic (n = 9) and hemorrhagic (n = 1) stroke during the last 2 months to 4 years. Online recognition of movement imagination was done by a classifier with a brain computer interface. An exo-skeleton supported passive movements in a paretic hand managed by the brain-computer interface. During 2 weeks the patients had 10 sessions of 45–90 minute duration each. For control, we used data from 5 stroke patients who, in addition to their standard treatment, underwent an imitation of rehabilitation procedures without movement imagination and feedback. To assess efficacy of treatment, we used a modified Ashworth scale, Fugl-Meyer scale, test for evaluation of hand functions ARAT, British scale for assessment of muscle force MRC-SS. Level of everyday activity and working ability was measured with a modified Rankin scale and Bartel index. Cognitive functions were assessed with Schulte tables. Results: Online recognition of movement imagination according to desynchronization of μ rhythm was registered in 50–75% of patients. All patients reported a subjective improvement of motor functions and working ability. Positive results for at least one parameter were observed in all patients; however, there were no significant difference between the parameters before and after rehabilitation procedures, excluding cognitive functions (degree of warming-up, p < 0.02). Conclusion: In post stroke patients, the use of movement imagination, brain-computer interface and exo-skeleton does not seem to affect the rehabilitation process negatively. In all cases, some positive results were achieved in motor recovery, as well as in working ability and daily activity. The results of the rehabilitation procedure are promising; however, the study should be continued.

Mind-Wandering Tends to Occur under Low Perceptual Demands during Driving

Fluctuations in attention behind the wheel poses a significant risk for driver safety. During transient periods of inattention, drivers may shift their attention towards internally-directed thoughts or feelings at the expense of staying focused on the road. This study examined whether increasing task difficulty by manipulating involved sensory modalities as the driver detected the lane-departure in a simulated driving task would promote a shift of brain activity between different modes of processing, reflected by brain network dynamics on electroencephalographic sources. Results showed that depriving the driver of salient sensory information imposes a relatively more perceptually-demanding task, leading to a stronger activation in the task-positive network. When the vehicle motion feedback is available, the drivers may rely on vehicle motion to perceive the perturbations, which frees attentional capacity and tends to activate the default mode network. Such brain network dynamics could have major implications for understanding fluctuations in driver attention and designing advance driver assistance systems.

Advancing 3D Single Molecule Tracking by Time-Gating and Fast Simultaneous Spinning Disk Imaging for Contextual Information

Three dimensional single molecule tracking is an essential technique for investigating biomolecular trafficking and signaling. Here we report the latest improvements to our confocal based tracking system. First, we added a Nipkow spinning disk imaging system. This system takes images of the plane of the tracked molecule, thus yielding contextual information about the molecule's environment. Laser illumination is used for the spinning disk system which together with an acousto-optic modulator allows for exposure times of 10 to 50 ms, thus greatly minimizing motion blur effects due to the feedback motion during tracking. Second, we introduced real-time photon time-gating to our tracking setup which allows tracking using only those photons detected within a defined time-window relative to the excitation laser pulse. Gating out detected photons within the first few nanoseconds after the excitation laser pulse greatly reduces Raman scattering and short lived fluorescence, and significantly increases the signal-to-noise ratio for tracked molecules with a long fluorescence lifetime. We also present a model for choosing the optimal time-gate based upon the fluorescence count rates and life times of the background and the tracked molecule.

Effects of Simulator Motion Feedback on Training of Skill-Based Control Behavior

This paper presents the results of a quasi-transfer-of-training experiment performed in the SIMONA Research Simulator at Delft University of Technology. The goal of the experiment was to quantify the effects of simulator motion feedback on the training of skill-based human operator control behavior using multimodal human operator modeling techniques. In the experiment, 24 task-naive participants, divided over two groups, were trained in performing a skill-based compensatory pitch tracking task. The first group was trained in a fixed-base setting and transferred to a moving-base condition; the second group trained with motion feedback and then transferred to the fixed-base condition. The group that received initial moving-base training showed quick adaptation of their control behavior upon transfer to the fixed-base setting and limited further learning. The group that trained in the fixed-base condition showed only limited transfer of their learned control strategy to the moving-base setting. After transfer, this group initially continued to rely exclusively on visual feedback, as indicated by very low identified motion response gains, and required an amount of moving-base training identical to the other group to develop multimodal control behavior. These results suggest that motion feedback is required for effective initial simulator-based training of skill-based manual control.

ServoFlight: pneumatically actuated full flight simulator

This contribution covers a new way to provide motion feedback in a full flight simulator. Seven pneumatic cylinders are utilized to actuate our 6 degrees-of-freedom motion platform. The motion platform being investigated is a parallel manipulator configured similarly to a Stewart platform with additional one redundant cylinder in the middle. The models of the pneumatic cylinder and the parallel manipulator are used to derive a dual control method. The dual control follows two different trajectories which are generated by a modified motion cueing algorithm and a force trajectory generator. The measurements are conducted on a full size build. The experiment results show that the platform performs well with a low total transport delay, therefore the platform is suitable to be used in a full flight simulator.

Sensor-based balance training with motion feedback in people with mild cognitive impairment.

Some individuals with mild cognitive impairment (MCI) experience not only cognitive deficits but also a decline in motor function, including postural balance. This pilot study sought to estimate the feasibility, user experience, and effects of a novel sensor-based balance training program. Patients with amnestic MCI (mean age 78.2 yr) were randomized to an intervention group (IG, n = 12) or control group (CG, n = 10). The IG underwent balance training (4 wk, twice a week) that included weight shifting and virtual obstacle crossing. Real-time visual/audio lower-limb motion feedback was provided from wearable sensors. The CG received no training. User experience was measured by a questionnaire. Postintervention effects on balance (center of mass sway during standing with eyes open [EO] and eyes closed), gait (speed, variability), cognition, and fear of falling were measured. Eleven participants (92%) completed the training and expressed fun, safety, and helpfulness of sensor feedback. Sway (EO, p = 0.04) and fear of falling (p = 0.02) were reduced in the IG compared to the CG. Changes in other measures were nonsignificant. Results suggest that the sensor-based training paradigm is well accepted in the target population and beneficial for improving postural control. Future studies should evaluate the added value of the sensor-based training compared to traditional training.

The Effects of Yaw and Sway Motion Cues in Curve Driving Simulation

This paper investigates the importance of yaw and sway motion cues in curve driving simulation. While such motion cues are known to enhance simulation realism, their function in supporting realistic driver behavior in simulators is still largely unknown. A human-in-the-loop curve driving experiment was performed in the SIMONA Research Simulator at TU Delft, in which eight participants were asked to follow a winding road’s center-line, while being subject to wind disturbances. Four motion conditions were tested: 1) no motion, 2) yaw only, 3) sway only, and 4) both yaw and sway; each was tested with 5 m and 100 m road preview for correspondence with earlier work. Results show that visual road preview is essential for adequate road-following. Although the effects of yaw and sway cues are much smaller, sway motion feedback allows for improved disturbance-rejection performance, while yaw motion feedback results in reduced control activity. These distinctly different effects suggest that both motion cues are important for evoking realistic driving behavior in simulators.

Age-related differences in control of a visuomotor coordination task: a preliminary study

[Purpose] The purpose of the current study was to examine age-related differences in control of a perception-action coordination skill. We adapted a visuomotor tracking experiment requiring various coordination patterns between a limb’s motion and an external signal. [Subjects and Methods] A total of 12 subjects (6 elderly and 6 young) voluntarily participated in the study. The experimental session consisted of 3 trials for 3 different relative phase patterns: 0°, 90°, and 180°, defined by the relationship between the online visual feedback of the joystick motion and the white dot signal. [Results] The 0° and 180° tracking patterns were stable compared with the 90° tracking pattern for both age groups. The present results also showed that the elderly subjects were less stable than were young subjects for all tracking patterns. [Conclusion] The intrinsic coordination dynamics predicted by the Haken-Kelso-Bunz (HKB) mathematical model did not change with age, whereas utilization of visual feedback information declined overall. Further research is needed regarding methods for increasing utilization of visual feedback information from the perspective of rehabilitation.

Maintenance of Alertness for a Car Driver by a Visual Feedback of Eye-gaze Motion and Arousal Level

Maintenance of Alertness for a Car Driver by a Visual Feedback of Eye-gaze Motion and Arousal Level

Принципы физиологии активности Н.А. Бернштейна в психологии восприятия и внимания: проблемы и перспективы

The paper reviews N.A. Berstein’s creative legacy in the development of Russian research in visual perception and attention starting from 1960s. It describes how these studies adopted four groups of concepts from Bernstein’s biocybernetics: first, the metaphor of movement construction; second, the concept of task; third, the idea of levels (hierarchy) of movement construction; and fourth, the principle of feedback or circular control of movement. The paper also discusses modern Russian level-based concepts of perception and attention that directly originated from Bernstein’s ideas. It traces the parallels between these concepts in Russian psychology of perception and the concepts of regulation of perception activity existing in European and American cognitive neuropsychology and modern neuroscience, such as ascending and descending processes in visual information processing and the hypothesis of predictive coding. The paper concludes with a discussion of the possibilities for empirically testing the models of perception and visual attention based on N.A. Bernstein’s ideas using modern methods of cognitive science. M. Falikman’s research was supported by the NRU HSE Fundamental Research Program.

Development of Motion Feedback System with Functional Electrical Stimulation Using Electromyogram

Development of Motion Feedback System with Functional Electrical Stimulation Using Electromyogram

Human Crossfeed in Dual-Axis Manual Control with Motion Feedback

While many realistic manual control tasks require human operators to control multiple degrees-of-freedomsimultaneously, our understanding of such multi-axis manual control has not moved far beyond considering it simply as the control of multiple fully-independent axes. This investigation aims to further our understanding of multi-axis control by focusing on one phenomenon that is known to occur in such tasks: crossfeed. Crossfeed occurs when operators’ inputs in one controlled axis feed into another controlled degree-of-freedom, thereby affecting overall control performance. A human-in-the-loop experiment, in which operators performed a dual-axis aircraft roll and pitch tracking task with physical motion feedback, was conducted in the SIMONA Research Simulator at TU Delft. Three conditions were tested: the full dual-axis control task, supplemented with reference single-axis roll and pitch tasks. Through the use of independent target and disturbance forcing function signals in both controlled axes, we were able to detect the presence of crossfeed in this dual-axis task from spectral analysis. Furthermore, these signals facilitated the objective identification of the dynamics of the crossfeed contribution, in parallel with estimating operators visual and motion responses. The crossfeed dynamics were found to resemble the well-known dynamics of human operators’ visual responses. The crossfeed contribution was found to explain up to 20% of the measured control inputs, thereby indicating that crossfeed can be a factor of significance in multi-axis manual control.

Interactive Sensor-Based Balance Training in Older Cancer Patients with Chemotherapy-Induced Peripheral Neuropathy: A Randomized Controlled Trial

Background: Cancer patients with chemotherapy-induced peripheral neuropathy (CIPN) have deficits in sensory and motor skills leading to inappropriate proprioceptive feedback, impaired postural control, and fall risk. Balance training programs specifically developed for CIPN patients are lacking. Objective: This pilot study investigated the effect of an interactive motor adaptation balance training program based on wearable sensors for improving balance in older cancer patients with CIPN. Methods: Twenty-two patients (age: 70.3 ± 8.7 years) with objectively confirmed CIPN [vibration perception threshold (VPT) >25 V] were randomized to either an intervention (IG) or a control (CG) group. The IG received interactive game-based balance training including repetitive weight shifting and virtual obstacle crossing tasks. Wearable sensors provided real-time visual/auditory feedback from the lower limb trajectory and allowed the perception of motor errors during each motor action. The CG received no exercise intervention and continued their normal activity. Outcome measures were changes in sway of ankle, hip, and center of mass (CoM) in both mediolateral and anteroposterior (AP) directions during 30-second balance tests with increasing task difficulty [i.e. standing in feet-closed position with eyes open (EO) and eyes closed (EC), and in semi-tandem position with EO] at baseline and after the intervention. Additionally, gait performance (speed, variability) and fear of falling [Falls Efficacy Scale-International (FES-I)] were measured. Results: Training was safe despite the participants' impaired health status, great severity of CIPN (VPT 49.6 ± 26.7 V), and great fear of falling (FES-I score 31.37 ± 11.20). After the intervention, sway of hip, ankle, and CoM was significantly reduced in the IG compared to the CG while standing in feet-closed position with EO (p = 0.010-0.022, except AP CoM sway) and in semi-tandem position (p = 0.008-0.035, except ankle sway). No significant effects were found for balance with EC, gait speed, and FES-I score (p > 0.05). Conclusions: This proof-of-concept study demonstrates that older cancer patients with CIPN can significantly improve their postural balance with specifically tailored, sensor-based exercise training. The training approach has potential as a therapy for improving CIPN-related postural control deficits. However, future studies comparing the proposed technology-based training with traditional balance training are required to evaluate the benefit of the interactive joint movement feedback.

Dependence on aspect ratio of symmetry breaking for oscillating foils: implications for flapping flight

Using two-dimensional direct numerical simulations, we investigate the flow in a fluid of kinematic viscosity ${\it\nu}$ and density ${\it\rho}$ around elliptical foils of density ${\it\rho}_{s}$ with major axis $c$ and minor axis $b$ for three different aspect ratios: $AR=b/c=1$ (a circle); $AR=0.5$; and $AR=0.1$. The vertical location of these foils $y_{s}(t)=A\sin (2{\rm\pi}f_{0}t)$ oscillates with amplitude $A$ and frequency $f_{0}$ in two distinct ways: ‘pure’ oscillation, where the foils are constrained to remain in place; and ‘flying’ oscillation, where horizontal motion is allowed. We simulate the flow for a range of the two appropriate control parameters, the non-dimensional amplitude, or Keulegan–Carpenter number $KC=2{\rm\pi}A/c$, and the non-dimensional frequency, or Stokes number ${\it\beta}=f_{0}c^{2}/{\it\nu}$. We observe three distinct patterns of asymmetry, labelled ‘S-type’ for synchronous asymmetry, ‘$\text{QP}_{\text{H}}$-type’ and ‘$\text{QP}_{\text{L}}$-type’ for quasi-periodic asymmetry at sufficiently high and sufficiently low (i.e. $AR=0.1$) aspect ratios, respectively. These patterns are separated at the critical locus in $KC$–${\it\beta}$ space by a ‘freezing point’ where the two incommensurate frequencies of the QP-type flows combine, and we show that this freezing point tends to occur at smaller values of $KC$ as $AR$ decreases. We find for the smallest aspect ratio case ($AR=0.1$) that the transition to asymmetry, for all values of $KC$, occurs for a critical value of an ‘amplitude’ Stokes number ${\it\beta}_{A}={\it\beta}(KC)^{2}=4{\rm\pi}^{2}f_{0}A^{2}/{\it\nu}\simeq 3$. The $\text{QP}_{\text{L}}$-type asymmetry for $AR=0.1$ is qualitatively different in physical and mathematical structure from the $\text{QP}_{\text{H}}$-type asymmetry at higher aspect ratio. The flows at the two ends of the ellipse become essentially decoupled from each other for the $\text{QP}_{\text{L}}$-type asymmetry, the two frequencies in the horizontal force signature being close to the primary frequency, rather than twice the primary frequency as in the $\text{QP}_{\text{H}}$-type asymmetry. Furthermore, the associated coefficients arising from a Floquet stability analysis close to the critical thresholds are profoundly different for low aspect ratio foils. Freedom to move slightly suppresses the transition to S-type asymmetry, and for certain parameters, if a purely oscillating foil subject to S-type asymmetry is released to move, flow symmetry is rapidly recovered due to the negative feedback of small horizontal foil motion. Conversely, for the ‘higher’ aspect ratios, the transition to $\text{QP}_{\text{H}}$-type asymmetry is encouraged when the foil is allowed to move, with strong positive feedback occurring between the shed vortices from successive oscillation cycles. For $AR=0.1$, freedom to move significantly encourages the onset of asymmetry, but the newly observed ‘primary’ $\text{QP}_{\text{L}}$-type asymmetry found for pure oscillation no longer occurs when the foil flies, with S-type asymmetry leading ultimately to locomotion at a constant speed occurring all along the transition boundary for all values of $KC$ and ${\it\beta}$.

How Do Gymnastics Coaches Provide Movement Feedback in Training?

The augmented feedback athletes receive from their coach is essential to improve performance. The aim of this exploratory study was to identify how coaches consider six feedback factors which might have an influence on the error-correction process. The six feedback factors were developed from a heuristic model of error correction, namely: a) visual perspective, b) visual experience, c) motor experience, d) personal relationship between coach and athlete, e) methodological knowledge, and f) biomechanical knowledge. Seventy-three active gymnastics coaches were asked to rank the importance and frequency of use of these six factors. In addition, we examined, in an exploratory way, if moderators can partly explain the weightings. Results show that the feedback factors can be ranked in importance and frequency of use, with methodological knowledge being the most essential in an error-correction situation. More studies are needed that explore if coaches should guide feedback according to their own motor experience. Gaze training for coaches is needed to improve the feedback and, ultimately, the movements.

Creative, Interactive Graphic Arts Movement Feedback Increases Movement Dose During Balance Therapy: A Pilot Study

Creative, Interactive Graphic Arts Movement Feedback Increases Movement Dose During Balance Therapy: A Pilot Study

Use of neonatal simulation models to assess competency in bag-mask ventilation.

Providing adequate bag-mask ventilation (BMV) is an essential skill for neonatal resuscitation. Often this skill is learned using simulation manikins. Currently, there is no means of measuring the adequacy of ventilation in simulated scenarios. Thus, it is not possible to ascertain proficiency. The first aim of this study was to measure the pressure generated during BMV as performed by providers with different skill levels and measure the impact of different feedback mechanisms. The second aim was to measure the pressure volume characteristics of two neonatal manikins to see how closely they reflect newborn lung mechanics. In Phase I to achieve the first aim, we evaluated BMV skills in different level providers including residents (n=5), fellows (n=5), neonatal nurse practitioners (n=5) and neonatologists (n=5). Each provider was required to provide BMV for 2-min epochs on the SimNewB (Laerdal), which had been instrumented to measure pressure-volume characteristics. In sequential 2-min epochs, providers were given different feedback including chest-wall movement alone compared to manometer plus chest-wall movement or chest-wall movement plus manometer plus laptop lung volume depiction. In Phase II of the study we measured pressure-volume characteristics in instrumented versions of the SimNewB (Laerdal) and NeoNatalie (Laerdal). In Phase I, all providers are compared with the neonatologists. All measurements of tidal volume (Vt) are below the desired 5 ml kg(-1). The greatest difference in Vt between the neonatologists and other providers occurs when only chest-wall movement is provided. A linear relationship is noted between Vt and PIP for both SimNewB and NeoNatalie. The compliance curves are not 'S-shaped' and are different between the two models (P<0.001). Phase I of this study demonstrates that the SimNewB with the feedback of chest-wall movement alone was the best method of distinguishing experienced from inexperienced providers during simulated BMV. Therefore this is likely to be the best method to ascertain proficiency. Phase II of the study shows that the currently available neonatal simulation manikins do not have pressure-volume characteristics that are reflective of newborn lung mechanics, which can result in suboptimal training.

Visual Feedback of Tongue Movement for Novel Speech Sound Learning.

Pronunciation training studies have yielded important information concerning the processing of audiovisual (AV) information. Second language (L2) learners show increased reliance on bottom-up, multimodal input for speech perception (compared to monolingual individuals). However, little is known about the role of viewing one's own speech articulation processes during speech training. The current study investigated whether real-time, visual feedback for tongue movement can improve a speaker's learning of non-native speech sounds. An interactive 3D tongue visualization system based on electromagnetic articulography (EMA) was used in a speech training experiment. Native speakers of American English produced a novel speech sound (/ɖ/; a voiced, coronal, palatal stop) before, during, and after trials in which they viewed their own speech movements using the 3D model. Talkers' productions were evaluated using kinematic (tongue-tip spatial positioning) and acoustic (burst spectra) measures. The results indicated a rapid gain in accuracy associated with visual feedback training. The findings are discussed with respect to neural models for multimodal speech processing.

Optic flow improves adaptability of spatiotemporal characteristics during split-belt locomotor adaptation with tactile stimulation.

Human locomotor adaptation requires feedback and feed-forward control processes to maintain an appropriate walking pattern. Adaptation may require the use of visual and proprioceptive input to decode altered movement dynamics and generate an appropriate response. After a person transfers from an extreme sensory environment and back, as astronauts do when they return from spaceflight, the prolonged period required for re-adaptation can pose a significant burden. In our previous paper, we showed that plantar tactile vibration during a split-belt adaptation task did not interfere with the treadmill adaptation however, larger overground transfer effects with a slower decay resulted. Such effects, in the absence of visual feedback (of motion) and perturbation of tactile feedback, are believed to be due to a higher proprioceptive gain because, in the absence of relevant external dynamic cues such as optic flow, reliance on body-based cues is enhanced during gait tasks through multisensory integration. In this study, we therefore investigated the effect of optic flow on tactile-stimulated split-belt adaptation as a paradigm to facilitate the sensorimotor adaptation process. Twenty healthy young adults, separated into two matched groups, participated in the study. All participants performed an overground walking trial followed by a split-belt treadmill adaptation protocol. The tactile group (TC) received vibratory plantar tactile stimulation only, whereas the virtual reality and tactile group (VRT) received an additional concurrent visual stimulation: a moving virtual corridor, inducing perceived self-motion. A post-treadmill overground trial was performed to determine adaptation transfer. Interlimb coordination of spatiotemporal and kinetic variables was quantified using symmetry indices and analyzed using repeated-measures ANOVA. Marked changes of step length characteristics were observed in both groups during split-belt adaptation. Stance and swing time symmetries were similar in the two groups, suggesting that temporal parameters are not modified by optic flow. However, whereas the TC group displayed significant stance time asymmetries during the post-treadmill session, such aftereffects were absent in the VRT group. The results indicated that the enhanced transfer resulting from exposure to plantar cutaneous vibration during adaptation was alleviated by optic flow information. The presence of visual self-motion information may have reduced proprioceptive gain during learning. Thus, during overground walking, the learned proprioceptive split-belt pattern is more rapidly overridden by visual input due to its increased relative gain. The results suggest that when visual stimulation is provided during adaptive training, the system acquires the novel movement dynamics while maintaining the ability to flexibly adapt to different environments.

Design space exploration of hardware platforms for interactive low latency movement sonification

Objective real-time acoustic feedback of complex movements arises in applications ranging from assisted training in sport to medical rehabilitation. This so called interactive movement sonification enables continuous feedback. Multiple studies have shown the benefit of sonification in movement relearning. Ubiquitous feedback requires a mobile and wearable hardware platform independent from any application. Therefore, in this paper a design space exploration regarding the design objectives power dissipation and latency is performed. Especially in sports or medical rehabilitation a low-latency feedback is crucial. Furthermore, low power dissipation mobile devices enable the utilization of the sonification technique within new applications. The most computational intensive task within the data-flow is real-time interactive sound synthesis. In contrast to commercial processors, customizable processors enable execution time reduction of critical sections by integration of application specific instruction set extensions. This paper aims to show the benefits from processor customization, as well as the location of consumer electronics devices within the design space.