Optimal Feedback Control Research Articles

Dissociable effects of urgency and evidence accumulation during reaching revealed by dynamic multisensory integration.

When making perceptual decisions, humans combine information across sensory modalities dependent on their respective uncertainties. However, it remains unknown how the brain integrates multisensory feedback during movement, and which factors besides sensory uncertainty influence sensory contributions. We performed two reaching experiments on healthy adults to investigate whether movement corrections to combined visual and mechanical perturbations scale with visual uncertainty. To describe the dynamics of multimodal feedback responses, we further varied movement time and visual feedback duration during the movement. The results of our first experiment show that the contribution of visual feedback decreased with uncertainty. Additionally, we observed a transient phase during which visual feedback responses were stronger during faster movements. In a follow-up experiment, we found that the contribution of vision increased more quickly during slow movements when we presented the visual feedback for a longer time. Muscle activity corresponding to these visual responses exhibited modulations with sensory uncertainty and movement speed ca. 100ms following the onset of the visual feedback. Using an optimal feedback control model, we show that the increased response to visual feedback during fast movements can be explained by an urgency-dependent increase in control gains. Further, the fact that a longer viewing duration increased the visual contributions suggests that the brain accumulates sensory information over time to estimate the state of the arm during reaching. Our results provide additional evidence concerning the link between reaching control and decision-making, both of which appear to be influenced by sensory evidence accumulation and response urgency.Significance statement The time-course of multisensory integration during movement, along with the factors influencing this process, still requires further investigation. Here, we tested how visual uncertainty, movement speed, and visual feedback duration influence reach corrections to combined visual and mechanical perturbations. Using an optimal feedback control model, we illustrate that the time-course of multimodal corrections follows the predictions of a Kalman filter which continuously weighs sensory feedback and internal predictions according to their reliability. Importantly, we further show that changes in movement speed led to urgency-dependent modulations of control gains. Our results corroborate previous research linking motor control and decision-making by highlighting that multisensory feedback responses depend on sensory evidence accumulation and response urgency in a similar way as decision-making processes.

Approximation of optimal feedback controls for stochastic reaction-diffusion equations

In this paper we present a method to approximate optimal feedback controls for stochastic reaction-diffusion equations. We derive two approximation results providing the theoretical foundation of our approach and allowing for explicit error estimates. The approximation of optimal feedback controls by neural networks is discussed as an explicit application of our method. We illustrate our findings in the case of a linear quadratic control problem with a numerical example.

A Multilinear HJB-POD Method for the Optimal Control of PDEs on a Tree Structure

Optimal control problems driven by evolutionary partial differential equations arise in many industrial applications and their numerical solution is known to be a challenging problem. One approach to obtain an optimal feedback control is via the Dynamic Programming principle. Nevertheless, despite many theoretical results, this method has been applied only to very special cases since it suffers from the curse of dimensionality. Our goal is to mitigate this crucial obstruction developing a version of dynamic programming algorithms based on a tree structure and exploiting the compact representation of the dynamical systems based on tensors notations via a model reduction approach. Here, we want to show how this algorithm can be constructed for general nonlinear control problems and to illustrate its performances on a number of challenging numerical tests introducing novel pruning strategies that improve the efficacy of the method. Our numerical results indicate a large decrease in memory requirements, as well as computational time, for the proposed problems. Moreover, we prove the convergence of the algorithm and give some hints on its implementation.

Stochastic Control Problems with State Reflections Arising from Relaxed Benchmark Tracking

This paper studies stochastic control problems motivated by optimal consumption with wealth benchmark tracking. The benchmark process is modeled by a combination of a geometric Brownian motion and a running maximum process, indicating its increasing trend in the long run. We consider a relaxed tracking formulation such that the wealth compensated by the injected capital always dominates the benchmark process. The stochastic control problem is to maximize the expected utility of consumption deducted by the cost of the capital injection under the dynamic floor constraint. By introducing two auxiliary state processes with reflections, an equivalent auxiliary control problem is formulated and studied, which leads to the Hamilton-Jacobi-Bellman equation with two Neumann boundary conditions. We establish the existence of a unique classical solution to the dual partial differential equation using some novel probabilistic representations involving the local time of some dual processes together with a tailor-made decomposition-homogenization technique. The proof of the verification theorem on the optimal feedback control can be carried out by some stochastic flow analysis and technical estimations of the optimal control. Funding: L. Bo and Y. Huang are supported by the National Natural Science Foundation of China [Grant 12471451], the Natural Science Basic Research Program of Shaanxi [Grant 2023-JC-JQ-05], the Shaanxi Fundamental Science Research Project for Mathematics and Physics [Grant 23JSZ010], and Fundamental Research Funds for the Central Universities [Grant 20199235177]. X. Yu is supported by the Hong Kong RGC General Research Fund (GRF) [Grant 15304122] and the Research Centre for Quantitative Finance at the Hong Kong Polytechnic University [Grant P0042708].

A Decoupled Feedback and Fast Norm Optimal Control approach for Normal and Superconducting RF Cavity disturbances compensation

Iterative learning controller (ILC)s of different types have been used in various engineering applications for system control in presence of repetitive disturbance, often encountered in batch control systems. A particle accelerator usually comprising of Radiofrequnecy (RF) cavities, when operated in pulsed mode processes particle beam similar to that in batch systems. However, an injected particle beam disturbs electric field inside RF cavity. In additions to beam loading disturbance encountered in all types of RF cavities, a superconducting cavity also encounters disturbance due to Lorentz force detuning. Under presence of such persisting disturbance/s, a Fast Norm Optimal ILC (FNOILC), helps achieve specified amplitude and phase stability of electric field, which results in beam of specified quality. A sophisticated control system helps achieve this objective. In this paper, FNOILC technique is explored for this purpose. Though, it has been reported for few accelerator facilities, in this work, a decoupling strategy is used in a feedback control loop, resulting into overall enhancement of control performance. Behavioral model of RF cavity is represented as dual input dual output system exhibiting two loops, found interacting due to nonzero cavity detuning. A decoupler is designed based on well known technique/s used in process control applications. Relative gain array element values are used to quantify interaction. In presence of beam loading disturbance, a FNOILC demonstrates an excellent control performance. A standard performance index has been used to assess control performance. It is found that use of decoupler in main feedback loop of RF cavity, enhances the resulting performance index. Besides, an ILC performance needs assessment with practical issues like model uncertainty, sensing delays, high detuning and observation noise. In this paper, most of conclusions are drawn based on large number of modeling and simulation studies carried out for lab prototype 650MHz cavity. Interesting studies like effects of low controller sampling frequency, feedback controller gain tuning, severe detuning, etc. are presented using a novel state plane representation. A zero phase filtering is simulated and proposed as an effective strategy when measurements have noise. This paper forms a case study which can be applied to any general system encountering repetitive disturbance. Additionally, paper also elaborates on FNOILC algorithm application for Lorentz force detuning compensation often encountered in superconducting cavity during pulse mode of operation.

Linear quadratic optimal control of stochastic 2-D Roesser models

This paper investigates the linear quadratic (LQ) optimal control problem for the stochastic two-dimensional (2-D) systems governed by Roesser models with multiplicative noise. The main contribution is to give the necessary and sufficient optimality condition by proposing a set of novel forward and backward stochastic partial difference equations (FBSPDE), and to further present the explicitly optimal feedback control laws on the finite horizon and on the infinite horizon based on the Riccati-like difference equations and the algebraic equation, respectively. Several numerical simulations are provided to illustrate the performance of the designed controllers.

Adaptive neural optimal tracking control for uncertain unmanned surface vehicle

This article proposes a novel adaptive neural optimal control scheme for the uncertain unmanned surface vehicle (USV) system. The proposed adaptive neural optimal control scheme is composed of a neural feedforward controller and a neural optimal feedback controller. The neural feedforward controller is designed by using neural networks (NNs) and the backstepping control design technique to make the unknown nonlinear dynamics of USV system to be stabilized. The neural optimal feedback controller is designed based on adaptive dynamic programming (ADP) theory. It is proved that the formulated adaptive neural optimal control scheme can achieve all signals in USV system are uniformly ultimately bounded (UUB) and the cost function is minimized. The simulation and comparison results demonstrate the effectiveness of the proposed optimal control scheme.

Feedback classification and optimal control with applications to the controlled Lotka–Volterra model

Let M be a σ-compact C ∞ manifold of dimension n ≥ 2 and consider a single-input control system: x ˙ ( t ) = X ( x ( t ) ) + u ( t ) Y ( x ( t ) ) , where X, Y are C ∞ vector fields on M. We prove that there exists an open set of pairs ( X , Y ) for the C ∞ -Whitney topology is such that they admit singular abnormal rays so that the spectrum of the projective singular Hamiltonian dynamics is feedback invariant. This result is applied to the controlled Lotka–Volterra dynamics, where such rays are related to shifted equilibria of the free dynamics.

Online optimal tracking control of unknown nonlinear singularly perturbed systems using single network adaptive critic with improved learning

This study is the first time devoted to seek an online optimal tracking solution for unknown nonlinear singularly perturbed systems based on single network adaptive critic (SNAC) design. Firstly, a novel identifier with more efficient parametric multi-time scales differential neural network (PMTSDNN) is developed to obtain the unknown system dynamics. Then, based on the identification results, the online optimal tracking controller consists of an adaptive steady control term and an optimal feedback control term is developed by using SNAC to solve the Hamilton–Jacobi–Bellman (HJB) equation online. New learning law considering filtered parameter identification error is developed for the PMTSDNN identifier and the SNAC, which can realize online synchronous learning and fast convergence. The Lyapunov approach is synthesized to ensure the convergence characteristics of the overall closed loop system consisting of the PMTSDNN identifier, the SNAC and the optimal tracking control policy. Three examples are provided to illustrate the effectiveness of the investigated method.

Optimal output feedback control for networked control systems with dual multi‐memoryless channels

AbstractThis paper delves into the output feedback control of systems with dual multi‐memoryless channels, where the data packets transmitted from the sensor to the controller (S/C) and from the controller to the actuator (C/A) are suffered from two different multi‐parallel Markovian fading channels. Two new multi‐state Markovian chains are introduced, by which the multiple fading channels system is transformed into a new jumping parameter system. Then a stochastic maximum principle is given along with an optimal state feedback controller for the case that the state is known. When the state is unknown, a Markov jump linear estimator is designed by using the completing square method. In this case, an output feedback controller is obtained by substituting the real system state with its estimation. Meanwhile, the separation principle is shown. Finally, the validity of these methods is verified by simulation examples.

Optimal feedback control stabilisation for fractional output in time-delayed distributed semilinear systems

This work focuses on the problem of stabilising the spatial fractional Riemann-Liouville output of order α ∈ [ 0 , 1 ] , for a class of distributed semi-linear systems with time delay evolving over the domain Ω. The main objective is to identify efficient feedback control strategies to achieve strong, weak and exponential stabilisdation of the fractional output. Then, we solve a minimisation fractional problem. To illustrate the practical implications of the stabilisation theorems presented, two examples of numerical simulations are provided.

Machine learning enhanced control co-design optimization of an immersion cooled battery thermal management system

The development of lithium-ion battery technology has ensured that battery thermal management systems are an essential component of the battery pack for next-generation energy storage systems. Using dielectric immersion cooling, researchers have demonstrated the ability to attain high heat transfer rates due to the direct contact between cells and the coolant. However, feedback control has not been widely applied to immersion cooling schemes. Furthermore, current research has not considered battery pack plant design when optimizing feedback control. Uncertainties are inherent in the cooling equipment, resulting in temperature and flow rate fluctuations. Hence, it is crucial to systematically consider these uncertainties during cooling system design to improve the performance and reliability of the battery pack. To fill this gap, we established a reliability-based control co-design optimization framework using machine learning for immersion cooled battery packs. We first developed an experimental setup for 21700 battery immersion cooling, and the experiment data were used to build a high-fidelity multiphysics finite element model. The model can precisely represent the electrical and thermal profile of the battery. We then developed surrogate models based on the finite element simulations in order to reduce computational cost. The reliability-based control co-design optimization was employed to find the best plant and control design for the cooling system, in which an outer optimization loop minimized the cooling system cost while an inner loop ensured battery pack reliability. Finally, an optimal cooling system design was obtained and validated, which showed a 90% saving in cooling system energy consumption.

Human navigation strategies and their errors result from dynamic interactions of spatial uncertainties

Goal-directed navigation requires continuously integrating uncertain self-motion and landmark cues into an internal sense of location and direction, concurrently planning future paths, and sequentially executing motor actions. Here, we provide a unified account of these processes with a computational model of probabilistic path planning in the framework of optimal feedback control under uncertainty. This model gives rise to diverse human navigational strategies previously believed to be distinct behaviors and predicts quantitatively both the errors and the variability of navigation across numerous experiments. This furthermore explains how sequential egocentric landmark observations form an uncertain allocentric cognitive map, how this internal map is used both in route planning and during execution of movements, and reconciles seemingly contradictory results about cue-integration behavior in navigation. Taken together, the present work provides a parsimonious explanation of how patterns of human goal-directed navigation behavior arise from the continuous and dynamic interactions of spatial uncertainties in perception, cognition, and action.

New Equilibria and Dynamic Structures Under Continuous Optimal Feedback Control

In this paper, the new equilibria realized by continuous optimal control inputs and the dynamic structure around them are studied. Using the Euler–Lagrange equation, which is a necessary condition for optimal control problems, the equations of motion of a dynamic system with optimal control inputs that minimize the quadratic cost function are described in terms of state and adjoint variables. Based on the equations of motion, equilibrium conditions are derived, and the properties of equilibria are analyzed for the two-body and Hill three-body problems. The stability and dynamic structure around unstable equilibria are also characterized to get insights into the properties of optimal trajectories.

A nonlinear optimal control approach for 3-DOF four-cable driven parallel robots

In this article, a nonlinear optimal control approach is proposed for the dynamic model of 3-DOF four-cable driven parallel robots (CDPR). To solve the associated nonlinear optimal control problem, the dynamic model of the 3-DOF cable-driven parallel robot undergoes approximate linearization around a temporary operating point that is recomputed at each time-step of the control method. The linearization relies on Taylor series expansion and on the associated Jacobian matrices. For the linearized state-space model of the 3-DOF cable-driven parallel robot a stabilizing optimal (H-infinity) feedback controller is designed. To compute the controller’s feedback gains an algebraic Riccati equation is repetitively solved at each iteration of the control algorithm. The stability properties of the control method are proven through Lyapunov analysis. The proposed nonlinear optimal control approach achieves fast and accurate tracking of reference setpoints under moderate variations of the control inputs and a minimum dispersion of energy.

LQR controller design for portable power supply based on flyback converter

Abstract The stability of the power repair equipment guarantees the reliability of the repair of electrical equipment. The following problems exist in the power supply of the emergency equipment: the load equipment randomly cuts in the power supply network; load equipment requires fast dynamic performance of the power supply. This challenges the stability of the portable power supply with the Flyback converter as the core. In this paper, an optimal feedback control strategy based on linear quadratic regulator (LQR) theory is proposed to improve the dynamic and steady-state performance of the Flyback converter. First, the state-averaged space model of the Flyback is derived and established. Second, an output voltage feedback integral controller is introduced to eliminate the steady-state error of the output voltage. Next, according to the LQR optimal control theory, the control model of the Flyback converter has been established, and the parameter design of the controller has been carried out by obtaining the optimal feedback gain matrix of the system. Finally, the simulation models are implemented with an output power of 120 W and a switching frequency of 50 kHz. The simulation results prove that the LQR controller provides superior performance than the traditional PI controller.

The neuron as a direct data-driven controller

In the quest to model neuronal function amid gaps in physiological data, a promising strategy is to develop a normative theory that interprets neuronal physiology as optimizing a computational objective. This study extends current normative models, which primarily optimize prediction, by conceptualizing neurons as optimal feedback controllers. We posit that neurons, especially those beyond early sensory areas, steer their environment toward a specific desired state through their output. This environment comprises both synaptically interlinked neurons and external motor sensory feedback loops, enabling neurons to evaluate the effectiveness of their control via synaptic feedback. To model neurons as biologically feasible controllers which implicitly identify loop dynamics, infer latent states, and optimize control we utilize the contemporary direct data-driven control (DD-DC) framework. Our DD-DC neuron model explains various neurophysiological phenomena: the shift from potentiation to depression in spike-timing-dependent plasticity with its asymmetry, the duration and adaptive nature of feedforward and feedback neuronal filters, the imprecision in spike generation under constant stimulation, and the characteristic operational variability and noise in the brain. Our model presents a significant departure from the traditional, feedforward, instant-response McCulloch-Pitts-Rosenblatt neuron, offering a modern, biologically informed fundamental unit for constructing neural networks.

Imbalanced optimal feedback motor control system in spinocerebellar ataxia type 3.

Human motor planning and control depend highly on optimal feedback control systems, such as the neocortex-cerebellum circuit. Here, diffusion tensor imaging was used to verify the disruption of the neocortex-cerebellum circuit in spinocerebellar ataxia type 3 (SCA3), and the circuit's disruption correlation with SCA3 motor dysfunction was investigated. This study included 45 patients with familial SCA3, aged 17-67 years, and 49 age- and sex-matched healthy controls, aged 21-64 years. Tract-based spatial statistics and probabilistic tractography was conducted using magnetic resonance images of the patients and controls. The correlation between the local probability of probabilistic tractography traced from the cerebellum and clinical symptoms measured using specified symptom scales was also calculated. The cerebellum-originated probabilistic tractography analysis showed that structural connectivity, mainly in the subcortical cerebellar-thalamo-cortical tract, was significantly reduced and the cortico-ponto-cerebellar tract was significantly stronger in the SCA3 group than in the control group. The enhanced tract was extended to the right lateral parietal region and the right primary motor cortex. The enhanced neocortex-cerebellum connections were highly associated with disease progression, including duration and symptomatic deterioration. Tractography probabilities from the cerebellar to parietal and sensorimotor areas were significantly negatively correlated with motor abilities in patients with SCA3. To our knowledge, this study is the first to reveal that disrupting the neocortex-cerebellum loop can cause SCA3-induced motor dysfunctions. The specific interaction between the cerebellar-thalamo-cortical and cortico-ponto-cerebellar pathways in patients with SCA3 and its relationship with ataxia symptoms provides a new direction for future research.

Neural network-based adaptive optimal tracking control for hypersonic morphing aircraft with appointed-time prescribed performance

In this paper, an adaptive optimal attitude tracking controller for hypersonic morphing aircraft with appointed-time prescribed performance is presented. A proper error transformation methodology is developed to ensure the performance constraints are met without knowing initial tracking conditions. Subsequently, an adaptive composite control scheme is devised for the transformed system, comprising a feedforward neuro-backstepping controller and a feedback optimal controller. The former is employed to convert the optimal tracking problem into an equivalent optimal regulation problem, while the latter is constructed using adaptive dynamic programming (ADP) with a single critic network to achieve optimality. A novel weight-tuning law for the critic network is introduced, incorporating a stability indicator and the concurrent learning technique. This approach relaxes the underlying restrictive conditions in ADP, improving the robustness and availability of the system. The stability of the closed-loop system is analyzed using the Lyapunov direct method. Finally, the effectiveness and improved performance of the proposed control scheme are validated through numerical simulations.

Design of reduced-order observer based steady state optimal linear-quadratic feedback controllers

In this paper we formulate and solve a problem encountered in engineering applications when a linear-quadratic (LQ) optimal feedback controller uses state estimates obtained via a reduced-order observer. Due to the use of state estimates instead of the actual state variables, the optimal quadratic performance is degraded in a pretty complex manner. In the paper, we show how to find the exact formula for the optimal performance degradation (optimal performance loss) in linear time invariant systems for the steady state case (infinite horizon optimization problem). The optimal performance loss is obtained in terms of solution of a reduced-order algebraic Lyapunov equation whose dimension is equal to the dimension of the reduced-order observer. The quantities that impact the performance criterion loss are identified. Practical examples (an inverted pendulum on a cart and an aircraft) show that the optimal performance loss can be very significant in some applications, and even very high in the presented linear-quadratic optimal control of an inverted pendulum problem. We have shown, using the derived formula, how the optimal performance loss can be considerably reduced by properly choosing the reduced-order observer initial conditions via the least square method.