Feedforward Information Research Articles

Different sensory information is used for state estimation when stationary or moving.

The accurate estimation of limb state is necessary for movement planning and execution. While state estimation requires both feedforward and feedback information, we focus here on the latter. Prior literature has shown that integrating visual and proprioceptive feedback improves estimates of static limb position. However, differences in visual and proprioceptive feedback delays suggest that multisensory integration could be disadvantageous when the limb is moving. We formalized this hypothesis by modeling feedback-based state estimation using the longstanding maximum likelihood estimation model of multisensory integration, which we updated to account for sensory delays. Our model predicted that the benefit of multisensory integration was largely lost when the limb was passively moving. We tested this hypothesis in a series of experiments in human subjects that compared the degree of interference created by discrepant visual or proprioceptive feedback when estimating limb position either statically at the end of the movement or dynamically at movement midpoint. In the static case, we observed significant interference: discrepant feedback in one modality systematically biased sensory estimates based on the other modality. However, no interference was seen in the dynamic case: participants could ignore sensory feedback from one modality and accurately reproduce the motion indicated by the other modality. Together, these findings suggest that the sensory feedback used to compute a state estimate differs depending on whether the limb is stationary or moving. While the former may tend toward multimodal integration, the latter is more likely to be based on feedback from a single sensory modality.Significance statement The estimation of limb state involves feedforward and feedback information. While estimation based on feedback has been well studied when the limb is stationary, it is unknown if similar sensory processing supports limb position estimates when moving. Using a computational model and behavioral experiments, we show that feedback-based state estimation may involve multisensory integration in the static case, but it is likely based on a single modality when the limb is moving. We suggest that this difference may stem from visual and proprioceptive feedback delays.

It's all in the timing: delayed feedback in autism may weaken predictive mechanisms during contour integration.

Humans rely on predictive and integrative mechanisms during visual processing to efficiently resolve incomplete or ambiguous sensory signals. Although initial low-level sensory data are conveyed by feedforward connections, feedback connections are believed to shape sensory processing through automatic conveyance of statistical probabilities based on prior exposure to stimulus configurations. Individuals with autism spectrum disorder (ASD) show biases in stimulus processing toward parts rather than wholes, suggesting their sensory processing may be less shaped by statistical predictions acquired through prior exposure to global stimulus properties. Investigations of illusory contour (IC) processing in neurotypical (NT) adults have established a well-tested marker of contour integration characterized by a robust modulation of the visually evoked potential (VEP)-the IC-effect-that occurs over lateral occipital scalp during the timeframe of the visual N1 component. Converging evidence strongly supports the notion that this IC-effect indexes a signal with significant feedback contributions. Using high-density VEPs, we compared the IC-effect in 6- to 17-yr-old children with ASD (n = 32) or NT development (n = 53). Both groups of children generated an IC-effect that was equivalent in amplitude. However, the IC-effect notably onset 21 ms later in ASD, even though initial VEP afference was identical across groups. This suggests that feedforward information predominated during perceptual processing for 15% longer in ASD compared with NT children. This delay in the feedback-dependent IC-effect, in the context of known developmental differences between feedforward and feedback fibers, suggests a potential pathophysiological mechanism of visual processing in ASD, whereby ongoing stimulus processing is less shaped by visual feedback.NEW & NOTEWORTHY Children with autism often present with an atypical visual perceptual style that emphasizes parts or details over the whole. Using electroencephalography (EEG), this study identifies delays in the visual feedback from higher-order sensory brain areas to primary sensory regions. Because this type of visual feedback is thought to carry information about prior sensory experiences, individuals with autism may have difficulty efficiently using prior experience or putting together parts into a whole to help make sense of incoming new visual information. This provides empirical neural evidence to support theories of disrupted sensory perception mechanisms in autism.

Different sensory information is used for state estimation when stationary or moving.

The accurate estimation of limb state is necessary for movement planning and execution. While state estimation requires both feedforward and feedback information, we focus here on the latter. Prior literature has shown that integrating visual and proprioceptive feedback improves estimates of static limb position. However, differences in visual and proprioceptive feedback delays suggest that multisensory integration could be disadvantageous when the limb is moving. We formalized this hypothesis by modeling feedback-based state estimation using the longstanding maximum likelihood estimation model of multisensory integration, which we updated to account for sensory delays. Our model predicted that the benefit of multisensory integration was largely lost when the limb was passively moving. We tested this hypothesis in a series of experiments in human subjects that compared the degree of interference created by discrepant visual or proprioceptive feedback when estimating limb position either statically at the end of the movement or dynamically at movement midpoint. In the static case, we observed significant interference: discrepant feedback in one modality systematically biased sensory estimates based on the other modality. However, no interference was seen in the dynamic case: participants could ignore sensory feedback from one modality and accurately reproduce the motion indicated by the other modality. Together, these findings suggest that the sensory feedback used to compute a state estimate differs depending on whether the limb is stationary or moving. While the former may tend toward multimodal integration, the latter is more likely to be based on feedback from a single sensory modality.

ViRgilites: Multilevel Feedforward for Multimodal Interaction in VR

Navigating the interaction landscape of Virtual Reality (VR) and Augmented Reality (AR) presents significant complexities due to the plethora of available input hardware and interaction modalities, compounded by spatially diverse visual interfaces. Such complexities elevate the likelihood of user errors, necessitating frequent backtracking. To address this, we introduce ViRgilites, a virtual guidance framework that delivers multi-level feedforward information covering the available interaction techniques as well as the future possibilities to interact with virtual objects, anticipating the interaction effects and how they fit with the overall user's goal. ViRgilites is engineered to facilitate task execution, empowering users to make informed decisions about action methodologies and alternative courses of action. This paper presents the architecture and functionality of ViRgilites and demonstrates its efficacy through evaluation with a formative user study.

Decoding Remapped Spatial Information in the Peri-Saccadic Period.

It has been suggested that, prior to a saccade, visual neurons predictively respond to stimuli that will fall in their receptive fields after completion of the saccade. This saccadic remapping process is thought to compensate for the shift of the visual world across the retina caused by eye movements. To map the timing of this predictive process in the brain, we recorded neural activity using electroencephalography during a saccade task. Human participants (male and female) made saccades between two fixation points while covertly attending to oriented gratings briefly presented at various locations on the screen. Data recorded during trials in which participants maintained fixation were used to train classifiers on stimuli in different positions. Subsequently, data collected during saccade trials were used to test for the presence of remapped stimulus information at the post-saccadic retinotopic location in the peri-saccadic period, providing unique insight into when remapped information becomes available. We found that the stimulus could be decoded at the remapped location ∼180 ms post-stimulus onset, but only when the stimulus was presented 100-200 ms before saccade onset. Within this range, we found that the timing of remapping was dictated by stimulus onset rather than saccade onset. We conclude that presenting the stimulus immediately before the saccade allows for optimal integration of the corollary discharge signal with the incoming peripheral visual information, resulting in a remapping of activation to the relevant post-saccadic retinotopic neurons.

Exploring the effect of fingertip aero-haptic feedforward cues in directing eyes-free target acquisition in VR

BackgroundThe sense of touch plays a crucial role in interactive behavior within virtual spaces, particularly when visual attention is absent. Although haptic feedback has been widely used to compensate for the lack of visual cues, the use of tactile information as a predictive feedforward cue to guide hand movements remains unexplored and lacks theoretical understanding. MethodsThis study introduces a fingertip aero-haptic rendering method to investigate its effectiveness in directing hand movements during eyes-free spatial interactions. The wearable device incorporates a multichannel micro-airflow chamber to deliver adjustable tactile effects on the fingertips. ResultsThe first study verified that tactile directional feedforward cues significantly improve user capabilities in eyes-free target acquisition and that users rely heavily on haptic indications rather than spatial memory to control their hands. A subsequent study examined the impact of enriched tactile feedforward cues on assisting users in determining precise target positions during eyes-free interactions, and assessed the required learning efforts. ConclusionsThe haptic feedforward effect holds great practical promise in eyeless design for virtual reality. We aim to integrate cognitive models and tactile feedforward cues in the future, and apply richer tactile feedforward information to alleviate users' perceptual deficiencies.

Improving the Feedforward Component for Recent Variants of Predictive Functional Control

A recent study demonstrated that the use of feedforward information with conventional Predictive Functional Control (PFC) leads to unexpected inconsistencies, with subsequent negative impacts on tuning and behaviour. A proposal was made to define the coincident point differently and shown to reduce the lag in the closed-loop PFC responses and applied to some systems with benign dynamics. Other recent work has looked at parameterisations of the future input to deal with challenging open-loop dynamics and significantly extended the range of problems for which PFC can be effective. This paper combines the two concepts, and thus proposes an algorithm that has both more effective and simple tuning than original PFC as well as being applicable to a range of challenging dynamics.

Distinct roles of delta- and theta-band neural tracking for sharpening and predictive coding of multi-level speech features during spoken language processing.

The brain tracks and encodes multi-level speech features during spoken language processing. It is evident that this speech tracking is dominant at low frequencies (<8 Hz) including delta and theta bands. Recent research has demonstrated distinctions between delta- and theta-band tracking but has not elucidated how they differentially encode speech across linguistic levels. Here, we hypothesised that delta-band tracking encodes prediction errors (enhanced processing of unexpected features) while theta-band tracking encodes neural sharpening (enhanced processing of expected features) when people perceive speech with different linguistic contents. EEG responses were recorded when normal-hearing participants attended to continuous auditory stimuli that contained different phonological/morphological and semantic contents: (1) real-words, (2) pseudo-words and (3) time-reversed speech. We employed multivariate temporal response functions to measure EEG reconstruction accuracies in response to acoustic (spectrogram), phonetic and phonemic features with the partialling procedure that singles out unique contributions of individual features. We found higher delta-band accuracies for pseudo-words than real-words and time-reversed speech, especially during encoding of phonetic features. Notably, individual time-lag analyses showed that significantly higher accuracies for pseudo-words than real-words started at early processing stages for phonetic encoding (<100 ms post-feature) and later stages for acoustic and phonemic encoding (>200 and 400 ms post-feature, respectively). Theta-band accuracies, on the other hand, were higher when stimuli had richer linguistic content (real-words > pseudo-words > time-reversed speech). Such effects also started at early stages (<100 ms post-feature) during encoding of all individual features or when all features were combined. We argue these results indicate that delta-band tracking may play a role in predictive coding leading to greater tracking of pseudo-words due to the presence of unexpected/unpredicted semantic information, while theta-band tracking encodes sharpened signals caused by more expected phonological/morphological and semantic contents. Early presence of these effects reflects rapid computations of sharpening and prediction errors. Moreover, by measuring changes in EEG alpha power, we did not find evidence that the observed effects can be solitarily explained by attentional demands or listening efforts. Finally, we used directed information analyses to illustrate feedforward and feedback information transfers between prediction errors and sharpening across linguistic levels, showcasing how our results fit with the hierarchical Predictive Coding framework. Together, we suggest the distinct roles of delta and theta neural tracking for sharpening and predictive coding of multi-level speech features during spoken language processing.

Distributed feedforward and feedback cortical processing supports human speech production.

Speech production is a complex human function requiring continuous feedforward commands together with reafferent feedback processing. These processes are carried out by distinct frontal and temporal cortical networks, but the degree and timing of their recruitment and dynamics remain poorly understood. We present a deep learning architecture that translates neural signals recorded directly from the cortex to an interpretable representational space that can reconstruct speech. We leverage learned decoding networks to disentangle feedforward vs. feedback processing. Unlike prevailing models, we find a mixed cortical architecture in which frontal and temporal networks each process both feedforward and feedback information in tandem. We elucidate the timing of feedforward and feedback-related processing by quantifying the derived receptive fields. Our approach provides evidence for a surprisingly mixed cortical architecture of speech circuitry together with decoding advances that have important implications for neural prosthetics.

Seeing does not mean processing: where we look and the visual information we rely on change independently as we learn a novel walking task.

People use vision to inform motor control strategies during walking. With practice performing a target stepping task, people shift their gaze farther ahead, transitioning from watching their feet contact the target to looking for future target locations. The shift in gaze focus suggests the role of vision in motor control changes from emphasizing feedback to feedforward control. The present study examines whether changing visual fixation location is accompanied by a similar change in reliance upon visual information. Twenty healthy young adults practiced stepping on moving targets projected on the surface of a treadmill. Periodically, participants' visual reliance was probed by hiding stepping targets which inform feedback or feedforward (targets < or > 1.5 steps ahead, respectively) motor control strategies. We calculated visual reliance as the increase in step error when targets were hidden. We hypothesized that with practice, participant reliance on feedback visual information would decrease and their reliance on feedforward visual information would increase. Contrary to our hypothesis, participants became significantly more reliant on feedback visual information with practice (p < 0.001) but their reliance on feedforward visual information did not change (p = 0.49). Participants' reliance on visual information increased despite looking significantly farther ahead with practice (p < 0.016). Together, these results suggest that participants fixated on feedback information less. However, changes in fixation pattern did not reduce their reliance upon feedback information as stepping performance still significantly decreased when feedback information was removed after training. These findings provide important context for how the role of vision in controlling walking changes with practice.

Fuzzy model predictive control for frequency regulation of temporary microgrids during load restoration

AbstractTemporary microgrids containing a high proportion of distributed renewable generations can offer great benefits during the recovery stage after large‐scale power system blackout, by picking up critical load and speeding up the entire grid recovery process. To solve the frequency security problem of temporary microgrids faced in the load restoration process, this paper proposes an adaptive fuzzy model predictive control (adaptive fuzzy‐MPC) strategy to obtain effective frequency regulation to suit the special characteristics of this process. First, the load restoration plan within a short time frame is regarded as the feedforward information and assists in the establishment of the MPC model. Then, considering the uncertainty of renewable distributed generations, the regulation resources are cooperated by an adaptive fuzzy controller which provides the weights of different regulation resources adopted in MPC. Specifically, the base weights of the energy storage system and renewable distributed generations are obtained from fuzzy rules, and then the weight for each regulation resource is modified by the designed adjustment rules. Simulation results illustrate that the proposed strategy can efficiently improve frequency performance while making various frequency regulation resources with different regulation capacities cooperate properly, which can further accelerate the load restoration progress and enhance system resilience.

Pathway-specific inputs to the superior colliculus support flexible responses to visual threat.

Behavioral flexibility requires directing feedforward sensory information to appropriate targets. In the superior colliculus, divergent outputs orchestrate different responses to visual threats, but the circuit organization enabling the flexible routing of sensory information remains unknown. To determine this structure, we focused on inhibitory projection (Gad2) neurons. Trans-synaptic tracing and neuronal recordings revealed that Gad2 neurons projecting to the lateral geniculate nucleus (LGN) and the parabigeminal nucleus (PBG) form two separate populations, each receiving a different set of non-retinal inputs. Inhibiting the LGN- or PBG-projecting Gad2 neurons resulted in opposing effects on behavior; increasing freezing or escape probability to visual looming, respectively. Optogenetic activation of selected inputs to the LGN- and PBG-projecting Gad2 cells predictably regulated responses to visual threat. These data suggest that projection-specific sampling of brain-wide inputs provides a circuit design principle that enables visual inputs to be selectively routed to produce context-specific behavior.

Enhancing Prediction of Forelimb Movement Trajectory through a Calibrating-Feedback Paradigm Incorporating RAT Primary Motor and Agranular Cortical Ensemble Activity in the Goal-Directed Reaching Task.

Complete reaching movements involve target sensing, motor planning, and arm movement execution, and this process requires the integration and communication of various brain regions. Previously, reaching movements have been decoded successfully from the motor cortex (M1) and applied to prosthetic control. However, most studies attempted to decode neural activities from a single brain region, resulting in reduced decoding accuracy during visually guided reaching motions. To enhance the decoding accuracy of visually guided forelimb reaching movements, we propose a parallel computing neural network using both M1 and medial agranular cortex (AGm) neural activities of rats to predict forelimb-reaching movements. The proposed network decodes M1 neural activities into the primary components of the forelimb movement and decodes AGm neural activities into internal feedforward information to calibrate the forelimb movement in a goal-reaching movement. We demonstrate that using AGm neural activity to calibrate M1 predicted forelimb movement can improve decoding performance significantly compared to neural decoders without calibration. We also show that the M1 and AGm neural activities contribute to controlling forelimb movement during goal-reaching movements, and we report an increase in the power of the local field potential (LFP) in beta and gamma bands over AGm in response to a change in the target distance, which may involve sensorimotor transformation and communication between the visual cortex and AGm when preparing for an upcoming reaching movement. The proposed parallel computing neural network with the internal feedback model improves prediction accuracy for goal-reaching movements.

A look into feedback neural computation upon collision selectivity

Physiological studies have shown that a group of locust’s lobula giant movement detectors (LGMDs) has a diversity of collision selectivity to approaching objects, relatively darker or brighter than their backgrounds in cluttered environments. Such diversity of collision selectivity can serve locusts to escape from attack by natural enemies, and migrate in swarm free of collision. For computational studies, endeavours have been made to realize the diverse selectivity which, however, is still one of the most challenging tasks especially in complex and dynamic real world scenarios. The existing models are mainly formulated as multi-layered neural networks with merely feed-forward information processing, and do not take into account the effect of re-entrant signals in feedback loop, which is an essential regulatory loop for motion perception, yet never been explored in looming perception. In this paper, we inaugurate feedback neural computation for constructing a new LGMD-based model, named F-LGMD to look into the efficacy upon implementing different collision selectivity. Accordingly, the proposed neural network model features both feed-forward processing and feedback loop. The feedback control propagates output signals of parallel ON/OFF channels back into their starting neurons, thus makes part of the feed-forward neural network, i.e. the ON/OFF channels and the feedback loop form an iterative cycle system. Moreover, the feedback control is instantaneous, which leads to the existence of a fixed point whereby the fixed point theorem is applied to rigorously derive valid range of feedback coefficients. To verify the effectiveness of the proposed method, we conduct systematic experiments covering synthetic and natural collision datasets, and also online robotic tests. The experimental results show that the F-LGMD, with a unified network, can fulfil the diverse collision selectivity revealed in physiology, which not only reduces considerably the handcrafted parameters compared to previous studies, but also offers a both efficient and robust scheme for collision perception through feedback neural computation.

Towards Strong AI with Analog Neural Chips

Applied AI chips with neural networks fail to capture and scale different forms of human intelligence. In this study, the definition of a strong AI system in hardware and architecture for building neuro memristive strong AI chips is proposed. The architecture unit consists of loop and hoop networks that are built on recurrent and feed forward information propagation concepts. Applying the principle that ’every brain is different’; we build a strong network that can take different structural and functional forms. The strong networks are building using hybrids of loop and hoop networks having generalization abilities, with higher levels of randomness incorporated to introduce greater flexibility in creating different neural architectures.

Cortico-cortical feedback engages active dendrites in visual cortex

Sensory processing in the neocortex requires both feedforward and feedback information flow between cortical areas1. In feedback processing, higher-level representations provide contextual information to lower levels, and facilitate perceptual functions such as contour integration and figure–ground segmentation2,3. However, we have limited understanding of the circuit and cellular mechanisms that mediate feedback influence. Here we use long-range all-optical connectivity mapping in mice to show that feedback influence from the lateromedial higher visual area (LM) to the primary visual cortex (V1) is spatially organized. When the source and target of feedback represent the same area of visual space, feedback is relatively suppressive. By contrast, when the source is offset from the target in visual space, feedback is relatively facilitating. Two-photon calcium imaging data show that this facilitating feedback is nonlinearly integrated in the apical tuft dendrites of V1 pyramidal neurons: retinotopically offset (surround) visual stimuli drive local dendritic calcium signals indicative of regenerative events, and two-photon optogenetic activation of LM neurons projecting to identified feedback-recipient spines in V1 can drive similar branch-specific local calcium signals. Our results show how neocortical feedback connectivity and nonlinear dendritic integration can together form a substrate to support both predictive and cooperative contextual interactions.

Robust Multi-Model Estimation for Reliable Relative Navigation Based on Observability and Abnormity Analysis

High-precision relative positioning and navigation is a fundamental requirement for many applications such as flight formation, spacecraft docking and collision avoidance. The main purpose of this paper is to develop a robust multi-model estimation algorithm for reliable navigation when there are abnormities of measurement and motion. In order to deal with these abnormities, we propose a quantitative evaluation method of relative navigation system by introducing the degree of observability (DoO) and the degree of abnormity (DoA). In addition, we design a feedforward information fusion and a feedback information allocation method based on DoO and DoA, and thus form a multi-model robust estimation algorithm. In order to testify the effectiveness and robustness of the proposed algorithm, a practical experiment with real data sets gathered in urban areas has been carried out. The results showed that the maximum relative positioning RMSE reduction ratio can reach 75%, and the maximum relative velocity RMSE reduction ratio can reach 51% compared with EKF. Therefore, the proposed method can guarantee the accuracy and robustness of relative navigation under abnormal conditions.

Integration of Feedforward and Feedback Information Streams in the Modular Architecture of Mouse Visual Cortex.

Radial cell columns are a hallmark feature of cortical architecture in many mammalian species. It has long been held, based on the lack of orientation columns, that such functional units are absent in rodent primary visual cortex (V1). These observations led to the view that rodent visual cortex has a fundamentally different network architecture than that of carnivores and primates. While columns may be lacking in rodent V1, we describe in this review that modular clusters of inputs to layer 1 and projection neurons in the layers below are prominent features of the mouse visual cortex. We propose that modules organize thalamocortical inputs, intracortical processing streams, and transthalamic communications that underlie distinct sensory and sensorimotor functions. Expected final online publication date for the Annual Review of Neuroscience, Volume 46 is July 2023. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

A Spring Compensation Method for a Low-Cost Biped Robot Based on Whole Body Control.

At present, the research and application of biped robots is more and more popular. The popularity of biped robots can be better promoted by improving the motion performance of low-cost biped robots. In this paper, the method of the Linear Quadratic Regulator (LQR) is used to track a robot's center of mass (COM). At the same time, the whole-body-control method and value function generated in the process of tracking COM are used to construct the quadratic programming (QP) model of a biped robot. Through the above method, the torque feedforward of the robot is obtained in the Drake simulation platform. The torque feedforward information of the robot is transformed into position feedforward information by spring compensation. In this paper, open loop control and spring compensation are used, respectively, to make the robot perform simple actions. Generally, after the compensation method of spring compensation is adopted, the roll angle and pitch angle of the upper body of the robot are closer to 0 after the robot performs an action. However, as the selected motion can introduce more forward and lateral motions, the robot needs more spring clearance compensation to improve performance. For improving the motion performance of a low-cost biped robot, the experimental results show that the spring compensation method is both reasonable and effective.

The Official Journal of ATTD Advanced Technologies &amp; Treatments for Diabetes Conference 22‐25 February 2023 I Berlin &amp; Online

The Official Journal of ATTD Advanced Technologies &amp; Treatments for Diabetes Conference 22‐25 February 2023 I Berlin &amp; Online