Optimal Feedback Control Model Research Articles

Optimal tuned outrigger-ATMD systems considering seismic and wind loads

This paper formulates the optimal feedback control model for the outrigger-active tuned mass damper (ATMD) system and investigates the influence of some input variables on the control effect while considering seismic and wind loads. The paper first deduces the joint equation of motion of the outrigger-ATMD system. Different control theories are then used to formulate the optimal control force, the optimal frequency tuning, and the optimal damping tuning. Based on such optimal tuned outrigger-ATMD systems, sensitivity analyses for variables from the ATMD and outrigger system are implemented. With many carefully selected schemes, the control effect and efficiency of the control model of the ATMD are verified in the frequency domain and time domain. It is observed that the fixed-points theory is a better choice for the ATMD tuning and the supplement damping in the outrigger system will suppress the control performance of the ATMD.

Mixed-horizon optimal feedback control as a model of human movement

Computational optimal feedback control (OFC) models in the sensorimotor control literature span a vast range of different implementations. Among the popular algorithms, finite-horizon, receding-horizon or infinite-horizon linear-quadratic regulators (LQR) have been broadly used to model human reaching movements. While these different implementations have their unique merits, all three have limitations in simulating the temporal evolution of visuomotor feedback responses. Here we propose a novel approach - a mixed-horizon OFC - by combining the strengths of the traditional finite-horizon and the infinite-horizon controllers to address their individual limitations. Specifically, we use the infinite-horizon OFC to generate durations of the movements, which are then fed into the finite-horizon controller to generate control gains. We then demonstrate the stability of our model by performing extensive sensitivity analysis of both re-optimisation and different cost functions. Finally, we use our model to provide a fresh look to previously published studies by reinforcing the previous results, providing alternative explanations to previous studies, or generating new predictive results for prior experiments.

Transient deactivation of dorsal premotor cortex or parietal area 5 impairs feedback control of the limb in macaques

We can generate goal-directed motor corrections with surprising speed, but their neural basis is poorly understood. Here, we show that temporary cooling of dorsal premotor cortex (PMd) impaired both spatial accuracy and the speed of corrective responses, whereas cooling parietal area 5 (A5) impaired only spatial accuracy. Simulations based on optimal feedback control (OFC) models demonstrated that "deactivation" of the control policy (reduction in feedback gain) and state estimation (reduction in Kalman gain) caused impairments similar to that observed for PMd and A5 cooling, respectively. Furthermore, combined deactivation of both cortical regions led to additive impairments of individual deactivations, whereas reducing the amount of cooling to PMd led to impairments in response speed but not spatial accuracy, both also predicted by OFC models. These results provide causal support that frontoparietal circuits beyond primary somatosensory and motor cortices are involved in generating goal-directed motor corrections.

The Cerebro-Cerebellum as a Locus of Forward Model: A Review.

This review surveys physiological, behavioral, and morphological evidence converging to the view of the cerebro-cerebellum as loci of internal forward models. The cerebro-cerebellum, the phylogenetically newest expansion in the cerebellum, receives convergent inputs from cortical, subcortical, and spinal sources, and is thought to perform the predictive computation for both motor control, motor learning, and cognitive functions. This predictive computation is known as an internal forward model. First, we elucidate the theoretical foundations of an internal forward model and its role in motor control and motor learning within the framework of the optimal feedback control model. Then, we discuss a neural mechanism that generates various patterns of outputs from the cerebro-cerebellum. Three lines of supporting evidence for the internal-forward-model hypothesis are presented in detail. First, we provide physiological evidence that the cerebellar outputs (activities of dentate nucleus cells) are predictive for the cerebellar inputs [activities of mossy fibers (MFs)]. Second, we provide behavioral evidence that a component of movement kinematics is predictive for target motion in control subjects but lags behind a target motion in patients with cerebellar ataxia. Third, we provide morphological evidence that the cerebellar cortex and the dentate nucleus receive separate MF projections, a prerequisite for optimal estimation. Finally, we speculate that the predictive computation in the cerebro-cerebellum could be deployed to not only motor control but also to non-motor, cognitive functions. This review concludes that the predictive computation of the internal forward model is the unifying algorithmic principle for understanding diverse functions played by the cerebro-cerebellum.

Time-to-Target Simplifies Optimal Control of Visuomotor Feedback Responses.

Visuomotor feedback responses vary in intensity throughout a reach, commonly explained by optimal control. Here, we show that the optimal control for a range of movements with the same goal can be simplified to a time-to-target dependent control scheme. We measure our human participants’ visuomotor responses in five reaching conditions, each with different hand or cursor kinematics. Participants only produced different feedback responses when these kinematic changes resulted in different times-to-target. We complement our experimental data with a range of finite and non-finite horizon optimal feedback control (OFC) models, finding that the model with time-to-target as one of the input parameters best replicates the experimental data. Overall, this suggests that time-to-target is a critical control parameter in online feedback control. Moreover, we propose that for a specific task and known dynamics, humans can instantly produce a control signal without any additional online computation allowing rapid response onset and close to optimal control.

Vestibular modulation of visuomotor feedback gains in reaching.

Humans quickly and sophisticatedly correct their movements in response to changes in the world, such as when reaching to a target that abruptly changes its location. The vigor of these movement corrections is time-dependent, increasing if the time left to make the correction decreases, which can be explained by optimal feedback control (OFC) theory as an increase of optimal feedback gains. It is unknown whether corrections for changes in the world are as sophisticated under full-body motion. For successful visually probed motor corrections during full-body motion, not only the motion of the hand relative to the body needs to be taken into account, but also the motion of the hand in the world should be considered, because their relative positions are changing. Here, in two experiments, we show that visuomotor feedback corrections in response to target jumps are more vigorous for faster passive full-body translational acceleration than for slower acceleration, suggesting that vestibular information modulates visuomotor feedback gains. Interestingly, these corrections do not demonstrate the time-dependent characteristics that body-stationary visuomotor feedback gains typically show, such that an optimal feedback control model fell short to explain them. We further show that the vigor of corrections generally decreased over the course of trials within the experiment, suggesting that the sensorimotor system adjusted its gains when learning to integrate the vestibular input into hand motor control.NEW & NOTEWORTHY Vestibular information is used in the control of reaching movements to world-stationary visual targets, while the body moves. Here, we show that vestibular information also modulates the corrective reach responses when the target changes position during the body motion: visuomotor feedback gains increase for faster body acceleration. Our results suggest that vestibular information is integrated into fast visuomotor control of reaching movements.

Experimental and theoretical study of velocity fluctuations during slow movements in humans

Moving smoothly is generally considered as a higher-order goal of motor control and moving jerkily as a witness of clumsiness or pathology, yet many common and well-controlled movements (e.g., tracking movements) have irregular velocity profiles with widespread fluctuations. The origin and nature of these fluctuations have been associated with the operation of an intermittent process but in fact remain poorly understood. Here we studied velocity fluctuations during slow movements, using combined experimental and theoretical tools. We recorded arm movement trajectories in a group of healthy participants performing back-and-forth movements at different speeds, and we analyzed velocity profiles in terms of series of segments (portions of velocity between 2 minima). We found that most of the segments were smooth (i.e., corresponding to a biphasic acceleration) and had constant duration irrespective of movement speed and linearly increasing amplitude with movement speed. We accounted for these observations with an optimal feedback control model driven by a staircase goal position signal in the presence of sensory noise. Our study suggests that one and the same control process can explain the production of fast and slow movements, i.e., fast movements emerge from the immediate tracking of a global goal position and slow movements from the successive tracking of intermittently updated intermediate goal positions. NEW & NOTEWORTHY We show in experiments and modeling that slow movements could result from the brain tracking a sequence of via points regularly distributed in time and space. Accordingly, slow movements would differ from fast movement by the nature of the guidance and not by the nature of control. This result could help in understanding the origin and nature of slow and segmented movements frequently observed in brain disorders.

Modeling and computation of energy efficiency management with emission permits trading

In this paper, we present an optimal feedback control model to deal with the problem of energy efficiency management. Especially, an emission permits trading scheme is considered in our model, in which the decision maker can trade the emission permits flexibly. We make use of the optimal control theory to derive a Hamilton-Jacobi-Bellman (HJB) equation satisfied by the value function, and then propose an upwind finite difference method to solve it. The stability of this method is demonstrated and the accuracy, as well as the usefulness, is shown by the numerical examples. The optimal management strategies, which maximize the discounted stream of the net revenue, together with the value functions, are obtained. The effects of the emission permits price and other parameters in the established model on the results have been also examined. We find that the influences of emission permits price on net revenue for the economic agents with different initial quotas are quite different. All the results demonstrate that the emission permits trading scheme plays an important role in the energy efficiency management.

A Comparison of Intention Estimation Methods for Decoder Calibration in Intracortical Brain-Computer Interfaces.

Recent reports indicate that making better assumptions about the user's intended movement can improve the accuracy of decoder calibration for intracortical brain-computer interfaces. Several methods now exist for estimating user intent, including an optimal feedback control model, a piecewise-linear feedback control model, ReFIT, and other heuristics. Which of these methods yields the best decoding performance? Using data from the BrainGate2 pilot clinical trial, we measured how a steady-state velocity Kalman filter decoder was affected by the choice of intention estimation method. We examined three separate components of the Kalman filter: dimensionality reduction, temporal smoothing, and output gain (speed scaling). The decoder's dimensionality reduction properties were largely unaffected by the intention estimation method. Decoded velocity vectors differed by <5% in terms of angular error and speed vs. target distance curves across methods. In contrast, the smoothing and gain properties of the decoder were greatly affected (> 50% difference in average values). Since the optimal gain and smoothing properties are task-specific (e.g. lower gains are better for smaller targets but worse for larger targets), no one method was better for all tasks. Our results show that, when gain and smoothing differences are accounted for, current intention estimation methods yield nearly equivalent decoders and that simple models of user intent, such as a position error vector (target position minus cursor position), perform comparably to more elaborate models. Our results also highlight that simple differences in gain and smoothing properties have a large effect on online performance and can confound decoder comparisons.

A Single, Continuously Applied Control Policy for Modeling Reaching Movements with and without Perturbation.

It has been debated whether kinematic features, such as the number of peaks or decomposed submovements in a velocity profile, indicate the number of discrete motor impulses or result from a continuous control process. The debate is particularly relevant for tasks involving target perturbation, which can alter movement kinematics. To simulate such tasks, finite-horizon models require two preset movement durations to compute two control policies before and after the perturbation. Another model employs infinite- and finite-horizon formulations to determine, respectively, movement durations and control policies, which are updated every time step. We adopted an infinite-horizon optimal feedback control model that, unlike previous approaches, does not preset movement durations or use multiple control policies. It contains both control-dependent and independent noises in system dynamics, state-dependent and independent noises in sensory feedbacks, and different delays and noise levels for visual and proprioceptive feedbacks. We analytically derived an optimal solution that can be applied continuously to move an effector toward a target regardless of whether, when, or where the target jumps. This single policy produces different numbers of peaks and "submovements" in velocity profiles for different conditions and trials. Movements that are slower or perturbed later appear to have more submovements. The model is also consistent with the observation that subjects can perform the perturbation task even without detecting the target jump or seeing their hands during reaching. Finally, because the model incorporates Weber's law via a state representation relative to the target, it explains why initial and terminal visual feedback are, respectively, less and more effective in improving end-point accuracy. Our work suggests that the number of peaks or submovements in a velocity profile does not necessarily reflect the number of motor impulses and that the difference between initial and terminal feedback does not necessarily imply a transition between open- and closed-loop strategies.

Robust Brain-Machine Interface Design Using Optimal Feedback Control Modeling and Adaptive Point Process Filtering.

Much progress has been made in brain-machine interfaces (BMI) using decoders such as Kalman filters and finding their parameters with closed-loop decoder adaptation (CLDA). However, current decoders do not model the spikes directly, and hence may limit the processing time-scale of BMI control and adaptation. Moreover, while specialized CLDA techniques for intention estimation and assisted training exist, a unified and systematic CLDA framework that generalizes across different setups is lacking. Here we develop a novel closed-loop BMI training architecture that allows for processing, control, and adaptation using spike events, enables robust control and extends to various tasks. Moreover, we develop a unified control-theoretic CLDA framework within which intention estimation, assisted training, and adaptation are performed. The architecture incorporates an infinite-horizon optimal feedback-control (OFC) model of the brain’s behavior in closed-loop BMI control, and a point process model of spikes. The OFC model infers the user’s motor intention during CLDA—a process termed intention estimation. OFC is also used to design an autonomous and dynamic assisted training technique. The point process model allows for neural processing, control and decoder adaptation with every spike event and at a faster time-scale than current decoders; it also enables dynamic spike-event-based parameter adaptation unlike current CLDA methods that use batch-based adaptation on much slower adaptation time-scales. We conducted closed-loop experiments in a non-human primate over tens of days to dissociate the effects of these novel CLDA components. The OFC intention estimation improved BMI performance compared with current intention estimation techniques. OFC assisted training allowed the subject to consistently achieve proficient control. Spike-event-based adaptation resulted in faster and more consistent performance convergence compared with batch-based methods, and was robust to parameter initialization. Finally, the architecture extended control to tasks beyond those used for CLDA training. These results have significant implications towards the development of clinically-viable neuroprosthetics.

Hedging your bets: intermediate movements as optimal behavior in the context of an incomplete decision.

Existing theories of movement planning suggest that it takes time to select and prepare the actions required to achieve a given goal. These theories often appeal to circumstances where planning apparently goes awry. For instance, if reaction times are forced to be very low, movement trajectories are often directed between two potential targets. These intermediate movements are generally interpreted as errors of movement planning, arising either from planning being incomplete or from parallel movement plans interfering with one another. Here we present an alternative view: that intermediate movements reflect uncertainty about movement goals. We show how intermediate movements are predicted by an optimal feedback control model that incorporates an ongoing decision about movement goals. According to this view, intermediate movements reflect an exploitation of compatibility between goals. Consequently, reducing the compatibility between goals should reduce the incidence of intermediate movements. In human subjects, we varied the compatibility between potential movement goals in two distinct ways: by varying the spatial separation between targets and by introducing a virtual barrier constraining trajectories to the target and penalizing intermediate movements. In both cases we found that decreasing goal compatibility led to a decreasing incidence of intermediate movements. Our results and theory suggest a more integrated view of decision-making and movement planning in which the primary bottleneck to generating a movement is deciding upon task goals. Determining how to move to achieve a given goal is rapid and automatic.

Mini-max feedback control as a computational theory of sensorimotor control in the presence of structural uncertainty.

We propose a mini-max feedback control (MMFC) model as a robust approach to human motor control under conditions of uncertain dynamics, such as structural uncertainty. The MMFC model is an expansion of the optimal feedback control (OFC) model. According to this scheme, motor commands are generated to minimize the maximal cost, based on an assumption of worst-case uncertainty, characterized by familiarity with novel dynamics. We simulated linear dynamic systems with different types of force fields–stable and unstable dynamics–and compared the performance of MMFC to that of OFC. MMFC delivered better performance than OFC in terms of stability and the achievement of tasks. Moreover, the gain in positional feedback with the MMFC model in the unstable dynamics was tuned to the direction of instability. It is assumed that the shape modulations of the gain in positional feedback in unstable dynamics played the same role as that played by end-point stiffness observed in human studies. Accordingly, we suggest that MMFC is a plausible model that predicts motor behavior under conditions of uncertain dynamics.

A Real-Time Brain-Machine Interface Combining Motor Target and Trajectory Intent Using an Optimal Feedback Control Design

Real-time brain-machine interfaces (BMI) have focused on either estimating the continuous movement trajectory or target intent. However, natural movement often incorporates both. Additionally, BMIs can be modeled as a feedback control system in which the subject modulates the neural activity to move the prosthetic device towards a desired target while receiving real-time sensory feedback of the state of the movement. We develop a novel real-time BMI using an optimal feedback control design that jointly estimates the movement target and trajectory of monkeys in two stages. First, the target is decoded from neural spiking activity before movement initiation. Second, the trajectory is decoded by combining the decoded target with the peri-movement spiking activity using an optimal feedback control design. This design exploits a recursive Bayesian decoder that uses an optimal feedback control model of the sensorimotor system to take into account the intended target location and the sensory feedback in its trajectory estimation from spiking activity. The real-time BMI processes the spiking activity directly using point process modeling. We implement the BMI in experiments consisting of an instructed-delay center-out task in which monkeys are presented with a target location on the screen during a delay period and then have to move a cursor to it without touching the incorrect targets. We show that the two-stage BMI performs more accurately than either stage alone. Correct target prediction can compensate for inaccurate trajectory estimation and vice versa. The optimal feedback control design also results in trajectories that are smoother and have lower estimation error. The two-stage decoder also performs better than linear regression approaches in offline cross-validation analyses. Our results demonstrate the advantage of a BMI design that jointly estimates the target and trajectory of movement and more closely mimics the sensorimotor control system.

Movement Duration, Fitts's Law, and an Infinite-Horizon Optimal Feedback Control Model for Biological Motor Systems

Optimization models explain many aspects of biological goal-directed movements. However, most such models use a finite-horizon formulation, which requires a prefixed movement duration to define a cost function and solve the optimization problem. To predict movement duration, these models have to be run multiple times with different prefixed durations until an appropriate duration is found by trial and error. The constrained minimum time model directly predicts movement duration; however, it does not consider sensory feedback and is thus applicable only to open-loop movements. To address these problems, we analyzed and simulated an infinite-horizon optimal feedback control model, with linear plants, that contains both control-dependent and control-independent noise and optimizes steady-state accuracy and energetic costs per unit time. The model applies the steady-state estimator and controller continuously to guide an effector to, and keep it at, target position. As such, it integrates movement control and posture maintenance without artificially dividing them with a precise, prefixed time boundary. Movement pace is determined by the model parameters, and the duration is an emergent property with trial-to-trial variability. By considering the mean duration, we derived both the log and power forms of Fitts's law as different approximations of the model. Moreover, the model reproduces typically observed velocity profiles and occasional transient overshoots. For unbiased sensory feedback, the effector reaches the target without bias, in contrast to finite-horizon models that systematically undershoot target when energetic cost is considered. Finally, the model does not involve backward and forward sweeps in time, its stability is easily checked, and the same solution applies to movements of different initial conditions and distances. We argue that biological systems could use steady-state solutions as default control mechanisms and might seek additional optimization of transient costs when justified or demanded by task or context.

Feedback-Controlled Parallel Point Process Filter for Estimation of Goal-Directed Movements From Neural Signals

Real-time brain-machine interfaces have estimated either the target of a movement, or its kinematics. However, both are encoded in the brain. Moreover, movements are often goal-directed and made to reach a target. Hence, modeling the goal-directed nature of movements and incorporating the target information in the kinematic decoder can increase its accuracy. Using an optimal feedback control design, we develop a recursive Bayesian kinematic decoder that models goal-directed movements and combines the target information with the neural spiking activity during movement. To do so, we build a prior goal-directed state-space model for the movement using an optimal feedback control model of the sensorimotor system that aims to emulate the processes underlying actual motor control and takes into account the sensory feedback. Most goal-directed models, however, depend on the movement duration, not known a priori to the decoder. This has prevented their real-time implementation. To resolve this duration uncertainty, the decoder discretizes the duration and consists of a bank of parallel point process filters, each combining the prior model of a discretized duration with the neural activity. The kinematics are computed by optimally combining these filter estimates. Using the feedback-controlled model and even a coarse discretization, the decoder significantly reduces the root mean square error in estimation of reaching movements performed by a monkey.

Unsupervised adaptation of brain-machine interface decoders

The performance of neural decoders can degrade over time due to non-stationarities in the relationship between neuronal activity and behavior. In this case, brain-machine interfaces (BMI) require adaptation of their decoders to maintain high performance across time. One way to achieve this is by use of periodical calibration phases, during which the BMI system (or an external human demonstrator) instructs the user to perform certain movements or behaviors. This approach has two disadvantages: (i) calibration phases interrupt the autonomous operation of the BMI and (ii) between two calibration phases the BMI performance might not be stable but continuously decrease. A better alternative would be that the BMI decoder is able to continuously adapt in an unsupervised manner during autonomous BMI operation, i.e., without knowing the movement intentions of the user. In the present article, we present an efficient method for such unsupervised training of BMI systems for continuous movement control. The proposed method utilizes a cost function derived from neuronal recordings, which guides a learning algorithm to evaluate the decoding parameters. We verify the performance of our adaptive method by simulating a BMI user with an optimal feedback control model and its interaction with our adaptive BMI decoder. The simulation results show that the cost function and the algorithm yield fast and precise trajectories toward targets at random orientations on a 2-dimensional computer screen. For initially unknown and non-stationary tuning parameters, our unsupervised method is still able to generate precise trajectories and to keep its performance stable in the long term. The algorithm can optionally work also with neuronal error-signals instead or in conjunction with the proposed unsupervised adaptation.

Correlations in state space can cause sub-optimal adaptation of optimal feedback control models

Control of our movements is apparently facilitated by an adaptive internal model in the cerebellum. It was long thought that this internal model implemented an adaptive inverse model and generated motor commands, but recently many reject that idea in favor of a forward model hypothesis. In theory, the forward model predicts upcoming state during reaching movements so the motor cortex can generate appropriate motor commands. Recent computational models of this process rely on the optimal feedback control (OFC) framework of control theory. OFC is a powerful tool for describing motor control, it does not describe adaptation. Some assume that adaptation of the forward model alone could explain motor adaptation, but this is widely understood to be overly simplistic. However, an adaptive optimal controller is difficult to implement. A reasonable alternative is to allow forward model adaptation to ‘re-tune’ the controller. Our simulations show that, as expected, forward model adaptation alone does not produce optimal trajectories during reaching movements perturbed by force fields. However, they also show that re-optimizing the controller from the forward model can be sub-optimal. This is because, in a system with state correlations or redundancies, accurate prediction requires different information than optimal control. We find that adding noise to the movements that matches noise found in human data is enough to overcome this problem. However, since the state space for control of real movements is far more complex than in our simple simulations, the effects of correlations on re-adaptation of the controller from the forward model cannot be overlooked.Electronic supplementary materialThe online version of this article (doi:10.1007/s10827-011-0350-z) contains supplementary material, which is available to authorized users.

An Optimal Feedback Control Framework for Grasping Objects with Position Uncertainty

As we move, the relative location between our hands and objects changes in uncertain ways due to noisy motor commands and imprecise and ambiguous sensory information. The impressive capabilities humans display for interacting and manipulating objects with position uncertainty suggest that our brain maintains representations of location uncertainty and builds compensation for uncertainty into its motor control strategies. Our previous work demonstrated that specific control strategies are used to compensate for location uncertainty. However, it is an open question whether compensation for position uncertainty in grasping is consistent with the stochastic optimal feedback control, mainly due to the difficulty of modeling natural tasks within this framework. In this study, we develop a stochastic optimal feedback control model to evaluate the optimality of human grasping strategies. We investigate the properties of the model through a series of simulation experiments and show that it explains key aspects of previously observed compensation strategies. It also provides a basis for individual differences in terms of differential control costs-the controller compensates only to the extent that performance benefits in terms of making stable grasps outweigh the additional control costs of compensation. These results suggest that stochastic optimal feedback control can be used to understand uncertainty compensation in complex natural tasks like grasping.

Online Adaptation and Over-Trial Learning in Macaque Visuomotor Control

When faced with unpredictable environments, the human motor system has been shown to develop optimized adaptation strategies that allow for online adaptation during the control process. Such online adaptation is to be contrasted to slower over-trial learning that corresponds to a trial-by-trial update of the movement plan. Here we investigate the interplay of both processes, i.e., online adaptation and over-trial learning, in a visuomotor experiment performed by macaques. We show that simple non-adaptive control schemes fail to perform in this task, but that a previously suggested adaptive optimal feedback control model can explain the observed behavior. We also show that over-trial learning as seen in learning and aftereffect curves can be explained by learning in a radial basis function network. Our results suggest that both the process of over-trial learning and the process of online adaptation are crucial to understand visuomotor learning.