Neural Optimal Control Research Articles

Adaptive neural optimal tracking control for uncertain unmanned surface vehicle

This article proposes a novel adaptive neural optimal control scheme for the uncertain unmanned surface vehicle (USV) system. The proposed adaptive neural optimal control scheme is composed of a neural feedforward controller and a neural optimal feedback controller. The neural feedforward controller is designed by using neural networks (NNs) and the backstepping control design technique to make the unknown nonlinear dynamics of USV system to be stabilized. The neural optimal feedback controller is designed based on adaptive dynamic programming (ADP) theory. It is proved that the formulated adaptive neural optimal control scheme can achieve all signals in USV system are uniformly ultimately bounded (UUB) and the cost function is minimized. The simulation and comparison results demonstrate the effectiveness of the proposed optimal control scheme.

Adaptive Finite-Time-Based Neural Optimal Control of Time-Delayed Wheeled Mobile Robotics Systems.

For nonlinear systems with uncertain state time delays, an adaptive neural optimal tracking control method based on finite time is designed. With the help of the appropriate LKFs, the time-delay problem is handled. A novel nonquadratic Hamilton-Jacobi-Bellman (HJB) function is defined, where finite time is selected as the upper limit of integration. This function contains information on the state time delay, while also maintaining the basic information. To meet specific requirements, the integral reinforcement learning method is employed to solve the ideal HJB function. Then, a tracking controller is designed to ensure finite-time convergence and optimization of the controlled system. This involves the evaluation and execution of gradient descent updates of neural network weights based on a reinforcement learning architecture. The semi-global practical finite-time stability of the controlled system and the finite-time convergence of the tracking error are guaranteed.

Delayed reinforcement learning converges to intermittent control for human quiet stance

The neural control of human quiet stance remains controversial, with classic views suggesting a limited role of the brain and recent findings conversely indicating direct cortical control of muscles during upright posture. Conceptual neural feedback control models have been proposed and tested against experimental evidence. The most renowned model is the continuous impedance control model. However, when time delays are included in this model to simulate neural transmission, the continuous controller becomes unstable. Another model, the intermittent control model, assumes that the central nervous system (CNS) activates muscles intermittently, and not continuously, to counteract gravitational torque. In this study, a delayed reinforcement learning algorithm was developed to seek optimal control policy to balance a one-segment inverted pendulum model representing the human body. According to this approach, there was no a-priori strategy imposed on the controller but rather the optimal strategy emerged from the reward-based learning. The simulation results indicated that the optimal neural controller exhibits intermittent, and not continuous, characteristics, in agreement with the possibility that the CNS intermittently provides neural feedback torque to maintain an upright posture.

Performance-prescribed optimal neural control for hypersonic vehicles considering disturbances: An adaptive dynamic programming approach

The precise trajectory tracking control of hypersonic vehicles plays a crucial role in secure flight. To address this challenge, this article develops an optimal neural control framework with prescribed performance constraints for height-velocity control tasks with disturbances. To this end, the control-oriented error model is firstly established and transformed into an affine nonlinear form to provide a foundation for the design of controller. Then, in order to balance performance constraints and convergence speed, a performance-constraint function is introduced to convert the constrained tracking error into an unconstrained one. After that, an improved sliding mode compensation observer is proposed by combining low-fidelity approximation data with high-fidelity compensation model for eliminating disturbances. The developed control scheme is organized by an adaptive dynamic programming (ADP) algorithm based on the actor-critic policy structure to achieve the near-optimal control strategy. Meanwhile, the designed neural network (NN) weight updating laws are provided to obtain the solution of Hamiltonian-Jacobi-Bellman (HJB) equation rapidly without high calculation consumption. Finally, the stability conditions of the closed-loop system are performed and ensure the entire framework is semi-global uniformly ultimately bounded (SGUUB), two simulation cases verify the effectiveness and superiority of the proposed approach.

Finite-Time Neural Optimal Control for Hypersonic Vehicle With AOA Constraint

Finite-Time Neural Optimal Control for Hypersonic Vehicle With AOA Constraint

Neural Optimal Control for Constrained Visual Servoing via Learning From Demonstration

This paper proposes a novel optimal control scheme for constrained image based visual servoing of a robot manipulator. For a robot manipulator with an eye-on-hand configuration, visibility constraint is an essential requirement to avoid servo failure, while robot’s actuator limits must also be satisfied. To ensure this, the constraints are modelled implicitly via learning the task and defining safe regions using expert human demonstrations via mixture of Dynamic Movement Primitives (DMPs). The visual servoing problem is then formulated as a closed-loop optimal control problem using these constraint model where a desired target (possibly time-varying) is obtained by acting upon the feedback from the real-time visual sensors. The visual servo control loop consists of a single network adaptive critic optimal tracking control scheme whose weights are tuned using Lyapunov stability criteria. The stability and the performance of the proposed control scheme is shown theoretically via Lyapunov approach and also verified experimentally using a seven degree of freedom (DOF) Franka Emika and six DOF Universal Robot (UR) 10 manipulator. The approach is also demonstrated on a use case scenarios in mock-up convenience store and warehouse setup. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The applications of robots in busy warehouses, healthcare sectors and convenience store, has a big societal impact. The scenarios in these real-world problems often consist of a dynamic environment. Therefore, sensor-based localization and planning, along with satisfying robot and task constraints are needed. These naturally adds up to the programming costs in addition to the robot platform and actuation. However, modern robots need intuitive and easy programming for more pervasive in a practical societal application. In our proposed method, this sensor-based planning with environmental constraints is modelled implicitly using the Programming by Demonstration framework. The user needs to catch the robot arm by his hand and teach the task at hand, and the framework captures the task and robot constraints while generalizing to new goals. This modelling, along with cost optimal controller, generates real-time constraint aware robot manipulation trajectories for the demonstrated task in dynamic environments.

Adaptive neural optimal tracking control of stochastic nonstrict-feedback nonlinear systems with output constraints

In this article, a novel adaptive optimal control method is proposed for constrained stochastic nonlinear systems in the form of nonstrict-feedback. The output constraints are handled by employing nonlinear mapping, and the controller is composed of feedforward and optimal parts. The feedforward controller is designed based on dynamic surface control technology with Gaussian properties, where a dynamic signal and linearly parametrized neural networks are employed to approximate the unknown dynamics and unavailable nonlinear functions, respectively. Using the mean value theorem and Young’s inequality, only one parameter must be online tweaked at each step. In the optimal part, the cost function is approximately obtained using adaptive dynamic programming for the optimal control law. All signals in the closed-loop systems are analytically proved to be semi-globally uniformly ultimately bounded in the probabilistic space. Numerical simulations are provided to verify the effectiveness of the proposed approach.

Adaptive actor‐critic neural optimal control for constrained nonstrict feedback nonlinear systems via command filter

AbstractThe actor‐critic neural optimal control is investigated for the state‐constrained nonlinear systems in the nonstrict feedback form with unmodeled dynamics in this paper. The filtering errors in the traditional dynamic surface control (DSC) are countervailed by the introduced compensation signals. Two design phases together determine the input: the feedforward input design and the near optimal input design. In the feedforward input design, a mapping rule is established to keep all the states in the finite range, and a first‐order adjunctive signal is designed to treat the unmodeled dynamics. In the near optimal input design, the cost function relying on the reconstructed error system is minimized by the near optimal input via adaptive dynamic programming (ADP). In the whole design processing, the unknown nonlinear uncertain parts are fitted by the radial basis function neural networks (RBFNNs). The stability analysis illustrates all the signals are bounded in the controlled system. Two simulation examples are employed to verify the theoretical findings.

Nonlinear neural control using feedback linearization explains task goals of central nervous system for trunk stability in sitting posture

Objective. Characterizing the task goals of the neural control system for achieving seated stability has been a fundamental challenge in human motor control research. This study aimed to experimentally identify the task goals of the neural control system for seated stability. Approach. Ten able-bodied young individuals participated in our experiments, which allowed us to measure their body motion and muscle activity during perturbed sitting. We used a nonlinear neuromechanical model of the seated human, along with a full-state feedback linearization approach and optimal control theory for identifying the neural control system and characterizing its task goals. Main results. We demonstrated that the neural feedback for trunk stability during seated posture uses angular position, velocity, acceleration, and jerk in a linearized space. The mean squared error between the predicted and measured motor commands was less than 0.6% among all trials and participants, with a median correlation coefficient of more than 0.9. Our identified optimal neural control primarily used trunk angular acceleration and near-minimum muscle activation to achieve seated stability while keeping the deviations of the trunk angular position and acceleration sufficiently small. Significance. Our proposed approach to neural control system identification relied on a performance criterion (e.g. cost function) explaining what the functional goal is and subsequently, finds the control law that leads to the best performance. Therefore, instead of assuming what control schemes the neural control might utilize (e.g. proportional-integral-derivative control), optimal control allows the motor task and the neuromechanical model to dictate a control law that best describes the physiological process. This approach allows for a mechanistic understanding of the neuromuscular mechanisms involved in seated stability and for inferring the task goals used by the neural control system to achieve the targeted motor behavior. Such neural control characterization can contribute to the development of objective balance evaluation tools and of bio-inspired assistive neuromodulation technologies.

Impulsive Neural Control to Schedule Antivirals and Immunomodulators for COVID-19.

New SARS-CoV-2 variants escaping the effect of vaccines are an eminent threat. The use of antivirals to inhibit the viral replication cycle or immunomodulators to regulate host immune responses can help to tackle the viral infection at the host level. To evaluate the potential use of these therapies, we propose the application of an inverse optimal neural controller to a mathematical model that represents SARS-CoV-2 dynamics in the host. Antiviral effects and immune responses are considered as the control actions. The variability between infected hosts can be large, thus, the host infection dynamics are identified based on a Recurrent High-Order Neural Network (RHONN) trained with the Extended Kalman Filter (EKF). The performance of the control strategies is tested by employing a Monte Carlo analysis. Simulation results present different scenarios where potential antivirals and immunomodulators could reduce the viral load.

Adaptive neural optimal control via command filter for nonlinear multi‐agent systems including time‐varying output constraints

AbstractIn this article, an optimal command‐filtered backstepping control approach is proposed for uncertain strict‐feedback nonlinear multi‐agent systems (MASs) including output constraints and unmodeled dynamics. One‐to‐one nonlinear mapping (NM) is utilized to recast constrained systems as corresponding unrestricted systems. A dynamical signal is applied to cope with unmodeled dynamics. Based on dynamic surface control (DSC), the feedforward controller is designed by introducing error compensating signals. The optimal feedback controller is produced applying adaptive dynamic programming (ADP) and integral reinforcement learning (IRL) techniques in which neural networks are utilized to approximate the relevant cost functions online with established weight updating laws. Therefore, the entire controller, including feedforward and feedback controllers, not only ensures that all signals in the closed‐loop systems are cooperative semi‐globally uniformly ultimately bounded (SGUUB) and the outputs maintain in the provided time‐varying constraints, but also makes sure that the cost functions achieve minimization. A simulation example is presented to illustrate the feasibility of the proposed control algorithm.

Command filter and dynamic surface control technology based adaptive optimal control of uncertain nonlinear systems in strict‐feedback form

AbstractIn this article, the problem of command filter and event‐triggered mechanism based adaptive neural optimal control is discussed for a class of nonlinear systems in the presence of unmodeled dynamics in strict‐feedback form. The overall controller design is composed of feedforward controller and feedback controller as well as event‐triggered controller design. In the first part, the feedforward controller is constructed by exploiting command filter and introducing the compensation error, and the first‐order filter in the classical dynamic surface control technology is replaced by utilizing the second‐order filter. In the second part, adaptive dynamic programming algorithm is used to estimate the unknown optimal index function and optimal control signal by the aid of the capability of neural networks. In the third part, based on the feedforward and feedback controllers, an adaptive event‐triggered control is developed to avoid the occurrence of Zeno behavior. All the signals in the controlled system are proved to be semi‐globally uniformly ultimately bounded through theoretical analysis. Two numerical examples are employed to verify the availability of the proposed method.

Publisher Correction to: Application of long short-term memory neural network and optimal control to variable-order fractional model of HIV/AIDS

Publisher Correction to: Application of long short-term memory neural network and optimal control to variable-order fractional model of HIV/AIDS

Optimal Synchronization of Unidirectionally Coupled FO Chaotic Electromechanical Devices With the Hierarchical Neural Network.

This article solves the problem of optimal synchronization, which is important but challenging for coupled fractional-order (FO) chaotic electromechanical devices composed of mechanical and electrical oscillators and electromagnetic filed by using a hierarchical neural network structure. The synchronization model of the FO electromechanical devices with capacitive and resistive couplings is built, and the phase diagrams reveal that the dynamic properties are closely related to sets of physical parameters, coupling coefficients, and FOs. To force the slave system to move from its original orbits to the orbits of the master system, an optimal synchronization policy, which includes an adaptive neural feedforward policy and an optimal neural feedback policy, is proposed. The feedforward controller is developed in the framework of FO backstepping integrated with the hierarchical neural network to estimate unknown functions of dynamic system in which the mentioned network has the formula transformation and hierarchical form to reduce the numbers of weights and membership functions. Also, an adaptive dynamic programming (ADP) policy is proposed to address the zero-sum differential game issue in the optimal neural feedback controller in which the hierarchical neural network is designed to yield solutions of the constrained Hamilton-Jacobi-Isaacs (HJI) equation online. The presented scheme not only ensures uniform ultimate boundedness of closed-loop coupled FO chaotic electromechanical devices and realizes optimal synchronization but also achieves a minimum value of cost function. Simulation results further show the validity of the presented scheme.

Optimal Operation for Reduced Energy Consumption of an Air Conditioning System Using Neural Inverse Optimal Control

For a comfortable thermal environment, the main parameters are indoor air humidity and temperature. These parameters are strongly coupled, causing the need to search for multivariable control alternatives that allow efficient results. Therefore, in order to control both the indoor air humidity and temperature for direct expansion (DX) air conditioning (A/C) systems, different controllers have been designed. In this paper, a discrete-time neural inverse optimal control scheme for trajectories tracking and reduced energy consumption of a DX A/C system is presented. The dynamic model of the plant is approximated by a recurrent high-order neural network (RHONN) identifier. Using this model, a discrete-time neural inverse optimal controller is designed. Unscented Kalman filter (UKF) is used online for the neural network learning. Via simulation the scheme is tested. The proposed approach effectiveness is illustrated with the obtained results and the control proposal performance against disturbances is validated.

Stabilization of a quadrotor system using an optimal neural network controller

Stabilization of a quadrotor system using an optimal neural network controller

Neural optimal tracking control of constrained nonaffine systems with a wastewater treatment application

In this paper, we aim to solve the optimal tracking control problem for a class of nonaffine discrete-time systems with actuator saturation. First, a data-based neural identifier is constructed to learn the unknown system dynamics. Then, according to the expression of the trained neural identifier, we can obtain the steady control corresponding to the reference trajectory. Next, by involving the iterative dual heuristic dynamic programming algorithm, the new costate function and the tracking control law are developed. Two other neural networks are used to estimate the costate function and approximate the tracking control law. Considering approximation errors of neural networks, the stability analysis of the proposed algorithm for the specific systems is provided by introducing the Lyapunov approach. Finally, via conducting simulation and comparison, the superiority of the developed optimal tracking method is confirmed. Moreover, the trajectory tracking performance of the wastewater treatment application is also involved for further verifying the proposed approach.

Optimal Neural Tracking Control with Metaheuristic Parameter Identification for Uncertain Nonlinear Systems with Disturbances

In this paper, we propose an inverse optimal neural control strategy for uncertain nonlinear systems subject to external disturbances. This control strategy is developed based on a neural observer for the estimation of unmeasured states and inverse optimal control theory for trajectory tracking. The stabilization of states along the desired trajectory is ensured via a control Lyapunov function. The optimal parameters of the control law are identified by different nature-inspired metaheuristic algorithms, namely: Ant Lion Optimizer, Grey Wolf Optimizer, Harris Hawks Optimization, and Whale Optimization Algorithm. Finally, a highly nonlinear biological system subject to parameter uncertainties and external disturbances (Activated Sludge Model) is proposed to validate the control strategy. Simulation results demonstrate that the control law with Ant Lion Optimizer outperforms the other optimization methods in terms of trajectory tracking in the presence of disturbances. The control law with Harris Hawks Optimization shows a better convergence of the neural states in presence of parameter uncertainty.

Adaptive neural optimal control of uncertain nonlinear systems with output constraints

In this paper, adaptive optimal control is proposed for a class of strict-feedback nonlinear systems with unmodeled dynamics and output constraints. The controller design procedure contains two parts. In the first part, to satisfy output constraints, nonlinear mapping is used to transform constrained system into a novel one without output constraints. A dynamical signal is utilized to deal with unmodeled dynamics. A feedforward controller is designed using the dynamics surface control technique. In the second part, an auxiliary dynamical system is introduced to optimize cost function, and neural-network based adaptive dynamic programming(ADP) is employed to approximate the optimal cost function and the optimal control law. It is proved that the closed-loop system is semi-globally uniformly ultimately bounded (SGUUB) and the output constraints are not triggered by theoretical analysis. Two simulation examples are provided to illustrate the effectiveness of the proposed scheme.

Epidural Stimulation Targeting Lower Urinary Tract Function in Spinally Intact and Transected Urethane‐Anesthetized Rats

Bladder management for conditions such as spinal cord injury (SCI) includes a combination of catheterization (either self‐intermittent or suprapubic) and pharmacological approaches, which targets maintenance but does not promote recovery of function. We have exciting data from multiple individuals receiving spinal cord epidural stimulation (scES) that is optimized for stepping and standing, in combination with task‐specific training, that shows improvements in lower urinary tract function. The objective of our current functional mapping pre‐clinical animal studies is to identify scES configurations (electrode location, amplitude and frequency) that can promote optimal neural control of urine storage (adequate bladder capacity at safe pressures without incontinence) and sufficient controlled emptying (high voiding efficiency at safe leak point pressures). Initial mapping was done in acute anesthetized terminal preparations during cystometry (bladder filling) using a modified Medtronic Specify 5‐6‐5 epidural array that is being used by our group in parallel human scES studies. The current data compares the response to scES in spinally intact and transected male and female rats 6 weeks post‐injury at T13‐L1 (level of sympathetic supply), L3‐4 (region with circuits related to external urethral sphincter control) and L6‐S1 (level of parasympathetic supply) spinal segments. Results indicate that scES storage effects and voiding efficiency were dependent upon presence/absence of spinal transection, frequency and intensity stimulation patterns, electrode location, and sex of the animals. Taken together these results suggest that the circuitry is dissociable and amenable to stimulation that seeks to promote either storage or voiding. Furthermore, our innovative approach and novel application of this Medtronic Specify 5‐6‐5 epidural device will generate functional maps that will lay the groundwork for expedient translation of this promising technology to individuals with SCI as well as provide a powerful tool for elucidating underlying circuits present in the lumbosacral cord that are involved in urogenital function.Support or Funding InformationNIH SPARC OT2OD024898