Biological Motor Control Research Articles

The neural optimal control hierarchy for motor control

Our empirical, neuroscientific understanding of biological motor systems has been rapidly growing in recent years. However, this understanding has not been systematically mapped to a quantitative characterization of motor control based in control theory. Here, we attempt to bridge this gap by describing the neural optimal control hierarchy (NOCH), which can serve as a foundation for biologically plausible models of neural motor control. The NOCH has been constructed by taking recent control theoretic models of motor control, analyzing the required processes, generating neurally plausible equivalent calculations and mapping them on to the neural structures that have been empirically identified to form the anatomical basis of motor control. We demonstrate the utility of the NOCH by constructing a simple model based on the identified principles and testing it in two ways. First, we perturb specific anatomical elements of the model and compare the resulting motor behavior with clinical data in which the corresponding area of the brain has been damaged. We show that damaging the assigned functions of the basal ganglia and cerebellum can cause the movement deficiencies seen in patients with Huntington's disease and cerebellar lesions. Second, we demonstrate that single spiking neuron data from our model's motor cortical areas explain major features of single-cell responses recorded from the same primate areas. We suggest that together these results show how NOCH-based models can be used to unify a broad range of data relevant to biological motor control in a quantitative, control theoretic framework.

Delayed feedback control requires an internal forward model

Biological motor control provides highly effective solutions to difficult control problems in spite of the complexity of the plant and the significant delays in sensory feedback. Such delays are expected to lead to non trivial stability issues and lack of robustness of control solutions. However, such difficulties are not observed in biological systems under normal operating conditions. Based on early suggestions in the control literature, a possible solution to this conundrum has been the suggestion that the motor system contains within itself a forward model of the plant (e.g., the arm), which allows the system to 'simulate' and predict the effect of applying a control signal. In this work, we formally define the notion of a forward model for deterministic control problems, and provide simple conditions that imply its existence for tasks involving delayed feedback control. As opposed to previous work which dealt mostly with linear plants and quadratic cost functions, our results apply to rather generic control systems, showing that any controller (biological or otherwise) which solves a set of tasks, must contain within itself a forward plant model. We suggest that our results provide strong theoretical support for the necessity of forward models in many delayed control problems, implying that they are not only useful, but rather, mandatory, under general conditions.

Learning variable impedance control

One of the hallmarks of the performance, versatility, and robustness of biological motor control is the ability to adapt the impedance of the overall biomechanical system to different task requirements and stochastic disturbances. A transfer of this principle to robotics is desirable, for instance to enable robots to work robustly and safely in everyday human environments. It is, however, not trivial to derive variable impedance controllers for practical high degree-of-freedom (DOF) robotic tasks. In this contribution, we accomplish such variable impedance control with the reinforcement learning (RL) algorithm PI 2 ( P olicy I mprovement with P ath I ntegrals). PI 2 is a model-free, sampling-based learning method derived from first principles of stochastic optimal control. The PI 2 algorithm requires no tuning of algorithmic parameters besides the exploration noise. The designer can thus fully focus on the cost function design to specify the task. From the viewpoint of robotics, a particular useful property of PI 2 is that it can scale to problems of many DOFs, so that reinforcement learning on real robotic systems becomes feasible. We sketch the PI 2 algorithm and its theoretical properties, and how it is applied to gain scheduling for variable impedance control. We evaluate our approach by presenting results on several simulated and real robots. We consider tasks involving accurate tracking through via points, and manipulation tasks requiring physical contact with the environment. In these tasks, the optimal strategy requires both tuning of a reference trajectory and the impedance of the end-effector. The results show that we can use path integral based reinforcement learning not only for planning but also to derive variable gain feedback controllers in realistic scenarios. Thus, the power of variable impedance control is made available to a wide variety of robotic systems and practical applications.

Biomimetic control for redundant and high degree of freedom limb systems: neurobiological modularity

We review the current understanding of modularity in biological motor control and its forms, and then relate this modularity to proposed modular control structures for biomimetic robots. We note the features that are different between the robotic and the biological 'designs' with features which have evolved by natural selection, and note those aspects of biology which may be counter-intuitive or unique to the biological controls as we currently understand them. Biological modularity can be divided into kinematic modularity comprised of strokes and cycles: primitives approximating a range of optimization criteria, and execution modularity comprised of kinetic motor primitives: muscle synergies recruited by premotor drives which are most often pulsatile, and which have the biomechanical effect of instantiating a visco-elastic force-field in the limb. The relations of these identified biological elements to kinematic and force-level motor primitives employed in robot control formulations are discussed.

Development of Adaptive Modular Active Leg (AMAL) using bipedal robotics technology

The objective of the work presented here is to develop a low cost active above knee prosthetic device exploiting bipedal robotics technology which will work utilizing the available biological motor control circuit properly integrated with a Central Pattern Generator (CPG) based control scheme. The approach is completely different from the existing Active Prosthetic devices, designed primarily as standalone systems utilizing multiple sensors and embedded rigid control schemes. In this research, first we designed a fuzzy logic based methodology for offering suitable gait pattern for an amputee, followed by formulating a suitable algorithm for designing a CPG, based on Rayleigh’s oscillator. An indigenous probe, Humanoid Gait Oscillator Detector (HGOD) has been designed for capturing gait patterns from various individuals of different height, weight and age. These data are used to design a Fuzzy inference system which generates most suitable gait pattern for an amputee. The output of the Fuzzy inference system is used for designing a CPG best suitable for the amputee. We then developed a CPG based control scheme for calculating the damping profile in real time for maneuvering a prosthetic device called AMAL (Adaptive Modular Active Leg). Also a number of simulation results are presented which show the stable behavior of knee and hip angles and determine the stable limit cycles of the network.

Stochastic optimization of a biologically plausible spino-neuromuscular system model

Simulations and modeling techniques are becoming increasingly important in understanding the behavior of biological systems. Detailed models help researchers answer questions in diverse areas such as the behavior of bacteria and viruses and aiding in the diagnosis and treatment of injuries and diseases. However, to yield meaningful biological behavior, biological simulations often include hundreds of parameters that correspond to biological components and characteristics. This paper demonstrates the effectiveness of genetic algorithms (GA) and particle swarm optimizer (PSO) based techniques in training biologically plausible behavior in a neuromuscular simulation of a biceps/triceps pair. The results are compared to human subjects during flexion/extension movements to show that these algorithms are effective in training biologically plausible behaviors on both neural and gross anatomical levels. Specific behaviors of interest that emerge include tonic tensions in both muscles during resting periods, biceps/triceps coactivation patterns, and recruitment-like behaviors. These are all fundamental characteristics of biological motor control and emerge without direct selection for these behaviors. This is the first time that all of these characteristic behaviors emerge in a model of this detail without direct selective pressure.

Modeling biological motor control for human locomotion with functional electrical stimulation

This paper develops a novel control system for functional electrical stimulation (FES) locomotion, which aims to generate normal locomotion for paraplegics via FES. It explores the possibility of applying ideas from biology to engineering. The neural control mechanism of the biological motor system, the central pattern generator, has been adopted in the control system design. Some artificial control techniques such as neural network control, fuzzy logic, control and impedance control are incorporated to refine the control performance. Several types of sensory feedback are integrated to endow this control system with an adaptive ability. A musculoskeletal model with 7 segments and 18 muscles is constructed for the simulation study. Satisfactory simulation results are achieved under this FES control system, which indicates a promising technique for the potential application of FES locomotion in future.

A forward‐propagation learning rule for neural inverse models in consideration of the correlation of propagated errors

AbstractWe have proposed the forward‐propagation rule (FP) as an inverse model learning scheme from the viewpoint of biological motor control. This learning scheme is based on a Newton‐like method, by which multilayered neural network can acquire an inverse model of the controlled object by a small number of iterative learning trials. There is a problem, however, that the learning procedure, which is characterized by estimation of the supervisor's signal for the input–output signal of the neuron, and also the solution of a linear multiple regression problem for updating the connection weights, is complicated, making it difficult to analyze the learning process. This paper introduces the correlation of the propagated error signal from the viewpoint of the maximum‐likelihood method in order to realize a goal‐directed learning, which has not hitherto been considered in FP, and extends the learning rule to the generalized least‐square method. As a result, it is clearly shown that the learning rule in FP is an approximate gradient method. The learning ability of the method is demonstrated by computer simulation. The proposed procedure contains a regularization term derived from the logarithmic likelihood, and the behavior after the convergence of learning exhibits a more stable tendency than in the conventional method. It is also shown that learning can be performed by a simplified method in which the error is simply propagated in the forward direction. © 2006 Wiley Periodicals, Inc. Syst Comp Jpn, 37(13): 54–66, 2006; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.20484

Heads up!

Unlike the human face, the neck has been largely overlooked in the computer graphics literature, this despite its complex anatomical structure and the important role that it plays in supporting the head in balance while generating the controlled head movements that are essential to so many aspects of human behavior. This paper makes two major contributions. First, we introduce a biomechanical model of the human head-neck system. Emulating the relevant anatomy, our model is characterized by appropriate kinematic redundancy (7 cervical vertebrae coupled by 3-DOF joints) and muscle actuator redundancy (72 neck muscles arranged in 3 muscle layers). This anatomically consistent biomechanical model confronts us with a challenging motor control problem, even for the relatively simple task of balancing the mass of the head in gravity atop the cervical spine. Hence, our second contribution is a novel neuromuscular control model for human head animation that emulates the relevant biological motor control mechanisms. Incorporating low-level reflex and high-level voluntary sub-controllers, our hierarchical controller provides input motor signals to the numerous muscle actuators. In addition to head pose and movement, it controls the tone of mutually opposed neck muscles to regulate the stiffness of the head-neck multibody system. Employing machine learning techniques, the neural networks within our neuromuscular controller are trained offline to efficiently generate the online pose and tone control signals necessary to synthesize a variety of autonomous movements for the behavioral animation of the human head and face.

Decoding sensory feedback from firing rates of afferent ensembles recorded in cat dorsal root ganglia in normal locomotion

Sensory feedback is required by biological motor control systems to maintain stability, respond to perturbations, and adapt. Similarly, motor neuroprostheses require feedback to provide natural and complete restoration of motor functions. In this paper, we show that ensemble firing rates from the body's mechanoreceptors can provide a natural source of kinematic state feedback and could be useful for prosthetic control. Single unit recordings from multiple primary afferent neurons were obtained during walking using multichannel electrode arrays implanted chronically in the L7 dorsal root ganglia of three cats. We typically recorded simultaneously from over 20-30 neurons during the first 7-14 days after surgery, but recordings gradually worsened thereafter. Histology indicates that a ring of inflammatory and connective tissues (100 microm thick) develops around each microelectrode and likely contributes to the degradation in recording quality. Accurate estimates of the hindlimb trajectory were made using a linear filter with inputs from only a few neurons highly correlated with limb kinematics. The coefficients for the linear filter were identified in a least-squares fit with 5-10 s of walking data (model training stage). The estimated and actual trajectories of separate walking data generally match well for walking at a range of speeds accounting for 63 +/- 22% (mean +/- S.D. for hip, knee, and ankle) of the variance in joint angle and 72 +/- 4% of the variance in joint angular velocities. These results indicate that a neural interface with primary sensory neurons in the dorsal root ganglion can provide accurate kinematic state information that may be useful for closed loop control of a neuroprosthesis.

Preflexes and internal models in biomimetic robot systems

The next generation of neuroprostheses, which are aimed at the restoration of natural movement of paralysed body parts or at the natural interaction with external devices, will be quite similar to biomimetic robot systems which attempt to duplicate the organization of the biological motor control system. In the paper, we review some of the organizing principles that have emerged in the last few years and might provide useful guidelines for a biomimetic design.

Consistency of Listing's law and reciprocal innervation with pseudo-inverse control of eye position in 3-D.

Pseudo-inverse kinematics, under which small movements are produced by the least possible sum square changes in motor command, has been proposed as a unifying principle for the elimination of redundancy in general biological motor control systems (Pellionsz 1984) and in particular in the oculomotor system (Daunicht 1988, 1991). We have noted elsewhere (Dean et al, 1999) that this principle is incomplete without first specifying a parameterisation of motor command space and we proposed that the relevant motor-command parameter is summed motor unit firing rate. Under this assumption we were able to show that pseudo-inverse control of the horizontal extraocular muscles is consistent with available motor pool firing rate data. In this paper we extend this result to three dimensions and six extraocular muscles, showing that pseudo-inverse control is consistent with published firing rate date for a realistic model of oculomotor kinematics. We suggest that pseudo-inverse control may represent a common currency for modular control of many degree of freedom systems while its implementation may be a consequences of the minimisation of a more ecologically relevant parameter such as post-saccadic retinal slip.

Learning from biological systems: modeling neural control

We have been pursuing a modular approach to modeling and investigating the neural control of posture and movement. A modular approach is a practical and natural way of investigating the function and structure of the complex central nervous system (CNS). The modules can include skeletal dynamics, muscle dynamics with recruitment and force-generation characteristics, basic spinal cord neural circuits responsible for the local stretch reflex, interneuronal networks that provide modulation of various reflexes, various structures of brain circuits, and control strategy decision-making centers. The anatomical detail and complexity of each module can vary depending on the specific scientific question or motor task to be investigated and the extent of information available on the hierarchy. To demonstrate the modular approach, we present two examples that illustrate how biological motor control tasks can be investigated in varying degrees of complexity and detail. One example shows that a model with detailed anatomical structure and dynamic properties has to be developed to investigate the detailed interactions among several sensory feedback loops in the spinal cord. The other example shows that when investigating whole-body behavior, a grosser structured model with lumped components is more tractable so issues such as the interaction of spinal cord regulation and upper CNS intelligent control can be investigated.

Dynamics and control problems in biology: some new challenges

It has been three years since the last special issue on Systems and Control Problems in Medicine appeared in the IEEE Transactions on Automatic Control. Since then, activity in this area has increased and the range of problems and future prospects for research has expanded. As one indication of the increased emphasis on systems and control approaches to medicine and biology, the IEEE Control Systems Society has established a Technical Committee on Biosystems and Control (http://www. ieeecss.org/TAB/Technical/biosystems/), and we are now pleased to present this special issue on this topic. This issue attempts to highlight some of the important problems in the biological realm that require mathematical modeling and analysis. A point to be emphasized is that biology and medicine represent rich sources of problems requiring dynamic modeling and systems analysis at various stages. For obvious reasons, we had to narrow the field of exposure in this issue, and we decided to highlight neuroscience and biomechanics for this purpose. The importance of a systems approach is exemplified by a recent trend combining these two fields in developing a new generation of neuroprostheses. The rapid growth in neuroengineering should attract the attention of control and systems engineers in that it promises many more control challenges—although we can only guess at the exact nature of these today. The five articles included in this issue represent ongoing research by the respective authors that lies at the interface between control systems, computational neuroscience, and biomechanics. The first article, by Nenadic and Ghosh, focuses on signal processing and control problems with a population of neurons and introduces rate coding as a basis for representing analog signals. The second article, by Doya et al., describes various learning algorithms in the cerebellum, the basal ganglia, and the cerebral cortex and emphasizes the role of predictive modeling in the decomposition of complex tasks. The third article, by He et al., illustrates a detailed neuromuscular model for simulating biological motor control tasks. It also describes a lumped, macroscopic model for postural control. Schovanec, in the fourth article, addresses musculotendon dynamics with application to ocular dynamics and stress analysis in skeletal structures. The final article, by Abbas and Gillette, discusses the application of functional neuromuscular stimulation to controlling the standing posture of a patient with spinal cord injury. Each of the articles includes a detailed bibliography for readers interested in probing further.

Increasing sensor flexibility through neuromodulation.

Both biological and man-made motor control networks require input from sensors to allow for modification of the motor program. Real sensory neurons are more flexible than typical robotic sensors because they are dynamic rather than static. The membrane properties of neurons and hence their excitability can be modified by the presence of neuromodulatory substances. In the case of a sensory neuron, this can change, in a functionally significant way, the code used to describe a stimulus. For instance, extension of the neuron's dynamic range or modification of its filtering characteristics can result. This flexibility has an apparent cost. The code used may be situation-dependent and hence difficult to interpret. To address this issue and to understand how neuromodulation is used effectively in a motor control network, I am studying the GPR2 stretch receptor in the crustacean stomatogastric nervous system. Several different neuromodulatory substances can modify its encoding properties. Comparisons of physiological and anatomical evidence suggest that neuromodulation can be effected both by GPR2 itself and by other neurons in the network. These results suggest that the analog of neuromodulation might be useful for improving sensor performance in an artificial motor control system.

Polyhedral mixture of linear experts for many-to-one mapping inversion and multiple controllers

Feed-forward control schemes require an inverse mapping of the controlled system. In adaptive systems this inverse mapping is learned from examples. The biological motor control is very redundant, as are many robotic systems, therefore the mapping is many-to-one and the inverse problem is ill posed. In this paper we present a novel architecture and algorithms for the approximation and inversion of many-to-one functions. The proposed architecture retains all the possible solutions available to the controller in real time. This is done by a modified mixture of experts architecture, where each expert is linear and more than a single expert may be assigned to the same input region. The learning is implemented by the hinging hyperplanes algorithm. The proposed architecture is described and its operation is illustrated for some simple cases. Finally, the virtue of redundancy and its exploitation by multiple controllers are discussed.

A model-independent definition of attractor behavior applicable to interactive tasks

Both in designing teleoperators or haptic interfaces and in fundamental biological motor control studies, it is important to characterize the motor commands and mechanical impedance responses of the operator (or subject). Although such a characterization is fundamentally impossible for isolated movements when these two aspects of motor behavior have similar time scales (as is the case with humans), it is nonetheless possible, if we are dealing with repeated movements, to measure a which is analogous to the current source in Norton-equivalent electrical circuits. We define the attractor to be this equivalent source and show that it rigorously embodies the notion of the attractor of a time-evolving system. We demonstrate that most previous attempts to test a controversial motor control hypothesis known as the equilibrium point or virtual trajectory hypothesis are based on inadequate models of the neuromuscular system, and we propose a model-independent means of testing the hypothesis based on a comparison of measurable attractor trajectories at different levels of the motor system. We present and demonstrate means of making such measurements experimentally and of assigning error bounds to the estimated trajectories.

A hierarchical foundation for models of sensorimotor control.

Successful performance of a sensorimotor task arises from the interaction of descending commands from the brain with the intrinsic properties of the lower levels of the sensorimotor system, including the dynamic mechanical properties of muscle, the natural coordinates of somatosensory receptors, the interneuronal circuitry of the spinal cord, and computational noise in these elements. Engineering models of biological motor control often oversimplify or even ignore these lower levels because they appear to complicate an already difficult problem. We modeled three highly simplified control systems that reflect the essential attributes of the lower levels in three tasks: acquiring a target in the face of random torque-pulse perturbations, optimizing fusimotor gain for the same perturbations, and minimizing postural error versus energy consumption during low- versus high-frequency perturbations. The emergent properties of the lower levels maintained stability in the face of feedback delays, resolved redundancy in over-complete systems, and helped to estimate loads and respond to perturbations. We suggest a general hierarchical approach to modeling sensorimotor systems, which better reflects the real control problem faced by the brain, as a first step toward identifying the actual neurocomputational steps and their anatomical partitioning in the brain.

Continuous attractors and oculomotor control

A recurrent neural network can possess multiple stable states, a property that many brain theories have implicated in learning and memory. There is good evidence for such multistability in the brainstem neural network that controls eye position. Because the stable states are arranged in a continuous dynamical attractor, the network can store a memory of eye position with analog neural encoding. Continuous attractors in model networks depend on precisely tuned positive feedback, and their robust maintenance requires mechanisms of synaptic plasticity. These ideas may have wider scope than just the oculomotor system. More generally, the internal models postulated by theories of biological motor control may be recurrent networks with continuous attractors.

Positive force feedback control of muscles.

This study was prompted by recent evidence for the existence of positive force feedback in feline locomotor control. Our aim was to establish some basic properties of positive force feedback in relation to load compensation, stability, intrinsic muscle properties, and interaction with displacement feedback. In human subjects, muscles acting about the wrist and ankle were activated by feedback-controlled electrical stimulation. The feedback signals were obtained from sensors monitoring force and displacement. The signals were filtered to mimic transduction by mammalian tendon organ and muscle spindle receptors. We found that when muscles under positive force feedback were loaded inertially, they responded in a stable manner with increased active force. The activation attenuated the muscle stretch (yield) that would otherwise occur in the absence of feedback. With enough positive force feedback gain, yield could actually reverse. This behavior, which we termed the affirming reaction, was reminiscent of the mammalian positive supporting reaction, a postural response elicited by contact of the foot with the ground. Muscles under positive force feedback remained stable, even when the loop gain (Gf) was set at levels of 2 or 3. In a linear system, if Gf > 1, instability occurs when the loop is closed. On further investigation, we found that Gf changed with joint angle: it declined as the load-bearing muscle actively shortened. We inferred that in closed-loop operation, the active muscles always shortened until Gf approached unity. In other words, the length-tension curve of active muscle ensures stability even when force-related excitation of motoneurons is very large. Concomitant negative displacement feedback reinforced and stabilized load compensation up to a certain gain, beyond which instability occurred. In further trials we included delays of up to 40 ms in the positive force feedback pathway, to model the delays recently described for tendon organ reflexes in cat locomotion. Contrary to expectations, this did not destabilize the loop. Indeed, when instability was deliberately evoked by setting displacement feedback gain high, delays in the positive force feedback pathway actually stabilized control. The stabilization of positive force feedback by inherent properties of the neuromuscular system increases the functional scope to be expected of feedback from force receptors in biological motor control. Our results provide a rationale for the delayed excitatory action of Ib heteronymous input on extensor motoneurons in cat locomotion.