Worst-case Disturbance Research Articles

Identification of cross-frequency interactions in compressible cavity flow using harmonic resolvent analysis

The resolvent analysis reveals the worst-case disturbances and the most amplified response in a fluid flow that can develop around a stationary base state. The recent work by Padovan et al. (J. Fluid Mech., vol. 900, 2020, A14) extended the classical resolvent analysis to the harmonic resolvent analysis framework by incorporating the time-varying nature of the base flow. The harmonic resolvent analysis can capture the triadic interactions between perturbations at two different frequencies through a base flow at a particular frequency. The singular values of the harmonic resolvent operator act as a gain between the spatiotemporal forcing and the response provided by the singular vectors. In the current study, we formulate the harmonic resolvent analysis framework for compressible flows based on the linearized Navier–Stokes equation (i.e. operator-based formulation). We validate our approach by applying the technique to the low-Mach-number flow past an airfoil. We further illustrate the application of this method to compressible cavity flows at Mach numbers of 0.6 and 0.8 with a length-to-depth ratio of $2$ . For the cavity flow at a Mach number of 0.6, the harmonic resolvent analysis reveals that the nonlinear cross-frequency interactions dominate the amplification of perturbations at frequencies that are harmonics of the leading Rossiter mode in the nonlinear flow. The findings demonstrate a physically consistent representation of an energy transfer from slow-evolving modes toward fast-evolving modes in the flow through cross-frequency interactions. For the cavity flow at a Mach number of 0.8, the analysis also sheds light on the nature of cross-frequency interaction in a cavity flow with two coexisting resonances.

NN-Based Reinforcement Learning Optimal Control for Inequality-Constrained Nonlinear Discrete-Time Systems With Disturbances.

Based on actor-critic neural networks (NNs), an optimal controller is proposed for solving the constrained control problem of an affine nonlinear discrete-time system with disturbances. The actor NNs provide the control signals and the critic NNs work as the performance indicators of the controller. By converting the original state constraints into new input constraints and state constraints, the penalty functions are introduced into the cost function, and then the constrained optimal control problem is transformed into an unconstrained one. Further, the relationship between the optimal control input and worst-case disturbance is obtained using the Game theory. With Lyapunov stability theory, the control signals are ensured to be uniformly ultimately bounded (UUB). Finally, the effectiveness of the control algorithms is tested through a numeral simulation using a third-order dynamic system.

On the Worst-Case Disturbance of an Oscillator with Quadratic Damping by an External Force with a Given Integral

On the Worst-Case Disturbance of an Oscillator with Quadratic Damping by an External Force with a Given Integral

Comparison of the Responses of a Nonlinear Oscillator to the Worst-Case Disturbance Consisting of Two Instantaneous Impacts and the Worst-Case Rectangular Pulse Disturbance

Comparison of the Responses of a Nonlinear Oscillator to the Worst-Case Disturbance Consisting of Two Instantaneous Impacts and the Worst-Case Rectangular Pulse Disturbance

On the Worst-Case Disturbance of an Oscillator with Quadratic Damping by an External Force with a Given Integral

On the Worst-Case Disturbance of an Oscillator with Quadratic Damping by an External Force with a Given Integral

Online and Robust Intermittent Motion Planning in Dynamic and Changing Environments.

In this article, we propose RRT-Q X∞ , an online and intermittent kinodynamic motion planning framework for dynamic environments with unknown robot dynamics and unknown disturbances. We leverage RRT X for global path planning and rapid replanning to produce waypoints as a sequence of boundary-value problems (BVPs). For each BVP, we formulate a finite-horizon, continuous-time zero-sum game, where the control input is the minimizer, and the worst case disturbance is the maximizer. We propose a robust intermittent Q-learning controller for waypoint navigation with completely unknown system dynamics, external disturbances, and intermittent control updates. We execute a relaxed persistence of excitation technique to guarantee that the Q-learning controller converges to the optimal controller. We provide rigorous Lyapunov-based proofs to guarantee the closed-loop stability of the equilibrium point. The effectiveness of the proposed RRT-Q X∞ is illustrated with Monte Carlo numerical experiments in numerous dynamic and changing environments.

Pareto Optimal Strategy Under H∞ Constraint for the Mean-Field Stochastic Systems in Infinite Horizon.

This article focuses on the mean-field linear-quadratic Pareto (MF-LQP) optimal strategy design for stochastic systems in infinite horizon, which is with the H∞ constraint when the system is disturbed by external interferences. The stochastic bounded real lemma (SBRL) with any initial state in infinite horizon is first investigated based on the stabilizing solution of the generalized algebraic Riccati equation (GARE). Then, by discussing the convexity of the cost functional, the stochastic indefinite MF-LQP control problem is defined and solved based on the MF-LQ theory and Pareto theory. When the worst case disturbance is considered in the collaborative multiplayer system, we show that the Pareto optimal strategy design with H∞ constraint [or robust Pareto optimal strategy, (RPOS)] can be given via solving two coupled GAREs. When the worst case disturbance and the Pareto efficient strategy work, all Pareto solutions are obtained by a generalized Lyapunov equation. Finally, a practical example shows that the obtained results are effective.

Two-player zero-sum game based neural critic tracking control for UAUV with unknown disturbance via backstepping method

In this work, a new tracking control scheme is developed for underactuated autonomous underwater vehicle(UAUV) with unknown disturbance. First, an error tracking system is constructed using backstepping method, which transforms the tracking control problem into the two-player zero-sum game problem. One player is the optimal controller which is designed to make the UAUV track the desired trajectory. The other player is the unknown disturbance that represent the overall damping effect of the UAUV. Then, the single critic network based online policy iteration algorithm is designed to get the optimal control law and the worst-case disturbance, which can achieve the near-optimal control performance and relax the requirements for initial admissible control conditions. In order to improve the converge velocity of tracking error, the weight update law are designed. Furthermore, a discount coefficient is introduced into the performance index due to the nonlinearity and the complexity of UAUV. In addition, the stability of the UAUV is analyzed based on the Lyapunov stability theory the system is guaranteed to be uniformly ultimately bounded (UUB). Finally, the effectiveness of the proposed method is demonstrated through the real-time simulations on the UAUV model.

Optimal feedback control law for automated vehicles in the presence of cyberattacks: A min–max approach

While automated vehicles (AVs) are expected to bring a wide range of benefits to future transportation systems, emerging AV technologies open a door for cyberattacks, where a select number of AVs are compromised to drive in an adversarial manner. This could result in network-wide increases in traffic congestion and energy consumption, degrading the performance of transportation systems. Hence, it is necessary to develop attack mitigation strategies for AVs as AVs gradually become a reality. However, this is rather challenging due to the lack of knowledge of adversaries. Limited prior studies have assumed attacks with a given probability distribution without considering their malicious intention, which is far from realistic due to the adversarial nature of potential attacks.In this study, we derive an optimal feedback control law for AVs in the presence of cyberattacks. Notably, attacks are only assumed to have a bounded magnitude without being subject to any specific statistical distribution, which is not only of theoretical interest but also relaxes the assumptions seen in prior studies. More importantly, to deal with lack of knowledge of malicious attacks, we, for the first time, formulate a min–max control problem to minimize the worst-case disturbance to traffic flow. Specifically, we present a mathematical framework for describing mixed-autonomy traffic involving AVs and human-driven vehicles (HVs), considering two typical types of attacks, namely false data injection attack on sensor measurements and malicious attack on AV control commands. Based on the framework presented, we derive a set of necessary conditions of optimality for the min–max control problem, based on which an iterative computational algorithm is developed for determining the optimal control (driving) strategy of AVs. The effectiveness of the proposed approach is demonstrated via extensive numerical simulations.

Adversarial Multiagent Output Containment Graphical Game With Local and Global Objectives for UAVs

Multiple leader and follower graphical games constitute challenging problems for aerospace and robotics applications. One of the challenges is to address the mutual interests among the followers with an optimal control point of view. In particular, the traditional approaches treat the output containment problem by introducing selfish followers where each follower only considers their own utility. In this article, we propose a differential output containment game over directed graphs where the mutual interests among the followers are addressed with an objective functional that also considers the neighboring agents. The obtained output containment error system results in a formulation where outputs of all followers prove to converge to the convex hull spanned by the outputs of leaders in a game optimal manner. The output containment problem is solved using the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathscr{H}_\infty$</tex-math></inline-formula> output feedback method, where the new necessary and sufficient conditions are presented. Another challenge is to design distributed Nash equilibrium control strategies for such games, which cannot be achieved with the traditional quadratic cost functional formulation. Therefore, a modified cost functional that provides both Nash and distributed control strategies in the sense that each follower uses the state information of its own and neighbors is presented. Furthermore, an <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathscr{L}_{2}$</tex-math></inline-formula> gain bound of the output containment error system that experiences worst-case disturbances with respect to the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathscr{H}_\infty$</tex-math></inline-formula> criterion is investigated. The proposed methods are validated by means of multiagent quadrotor unmanned aerial vehicles output containment game simulations.

Mixed [Formula: see text]-synthesis tracking control and disturbance rejection in a robotic digit of an impaired human hand for anthropomorphic coordination.

In a partially impaired anthropomorphic hand, maintaining the movement coordination of the robotic digits with the central nervous system (CNS) and natural digits is crucial for robust performance. A challenge in the control perspective of movement coordination of a human hand is finding methods robust to the disturbances in a well-posed control problem of a biomechanical model. We use visco-elastic dynamics in the human palm frame of reference to explore the biomechanics of movement coordination to solve this control problem. Our biomechanical model incorporates the time delay due to actuation force, parametric uncertainty, exogenous disturbances, and sensory noise to constitute a 21-degree of freedom model. A mixed [Formula: see text]-synthesis controller, considering the real parametric uncertainty, represents the CNS in the control paradigm. We consider the robotic finger's flexion movement when perturbed from the initial equilibrium. The controller provides feedback force at the joints to regulate the robotic finger movement. The index finger follows a reference trajectory of the joint angular position profile and stabilizes at a flexion angle of 1 rad/s at a time of 1 s. The main control objective is to keep the angular displacement of the finger joint constant when a disturbance force acts. We simulate the modeling scheme in MATLAB/ Simulink. The results demonstrate that our controller scheme is robust against the worst-case disturbance and achieves the desired performance value. Developing a biologically inspired neurophysiological controller with robust performance has many applications, including assistive rehabilitation devices, hand movement disorder diagnosis, and robotic manipulators.

Optimal [formula omitted] tracking control of nonlinear systems with zero-equilibrium-free via novel adaptive critic designs

In this paper, a novel adaptive critic control method is designed to solve an optimal H∞ tracking control problem for continuous nonlinear systems with nonzero equilibrium based on adaptive dynamic programming (ADP). To guarantee the finiteness of a cost function, traditional methods generally assume that the controlled system has a zero equilibrium point, which is not true in practical systems. In order to overcome such obstacle and realize H∞ optimal tracking control, this paper proposes a novel cost function design with respect to disturbance, tracking error and the derivative of tracking error. Based on the designed cost function, the H∞ control problem is formulated as two-player zero-sum differential games, and then a policy iteration (PI) algorithm is proposed to solve the corresponding Hamilton–Jacobi–Isaacs (HJI) equation. In order to obtain the online solution to the HJI equation, a single-critic neural network structure based on PI algorithm is established to learn the optimal control policy and the worst-case disturbance law. It is worth mentioning that the proposed adaptive critic control method can simplify the controller design process when the equilibrium of the systems is not zero. Finally, simulations are conducted to evaluate the tracking performance of the proposed control methods.

Global receptivity analysis: physically realizable input–output analysis

In the context of transition analysis, linear input–output analysis determines the worst-case disturbances to a laminar base flow based on a generic right-hand-side volumetric/boundary forcing term. The worst-case forcing is not physically realizable, and, to our knowledge, a generic framework for posing physically realizable worst-case disturbance problems is lacking. In natural receptivity analysis, disturbances are forced by matching (typically local) solutions within the boundary layer to outer solutions consisting of free-stream vortical, entropic and acoustic disturbances. We pose a scattering formalism to restrict the input forcing to a set of realizable disturbances associated with plane-wave solutions of the outer problem. The formulation is validated by comparing with direct numerical simulations of a Mach 4.5 flat-plate boundary layer. We show that the method provides insight into transition mechanisms by identifying those linear combinations of plane-wave disturbances that maximize energy amplification over a range of frequencies. We also discuss how the framework can be extended to accommodate scattering from shocks and in shock layers for supersonic flow.

Model-free adaptive optimal control policy for Markov jump systems: A value iterations algorithm

This article develops a model-free adaptive optimal control policy for discrete-time Markov jump systems. First, a two-player zero-sum game is formulated to obtain an optimal control policy that minimizes a cost function against the worst-case disturbance. Second, an action and mode-dependent value function is set up for zero-sum game to search such a policy with convergence guarantee rather than solving an optimization problem satisfying coupled algebraic Riccati equations. To be specific, motivated by the Bellman optimal principle, we develop an online value iterations algorithm to solve the zero-sum game, which is learning while controlling without any initial stabilizing policy. By this algorithm, we can achieve disturbance attenuation for Markov jump systems without knowledge of the system matrices. The adaptivity to slowly changing uncertainties can also be achieved due to the model-free feature and policy convergence. Finally, the effectiveness and practical potential of the algorithm are demonstrated by considering two numerical examples and a solar boiler system.

Pareto Optimal Strategy under H∞ Constraint for Discrete-Time Stochastic Systems

This paper investigates the Pareto optimal strategy of discrete-time stochastic systems under H∞ constraint, in which the weighting matrices of the weighted sum cost function can be indefinite. Combining the H∞ control theory with the indefinite LQ control theory, the generalized difference Riccati equations (GDREs) are obtained. By means of the solution of the GDREs, the Pareto optimal strategy with H∞ constraint is derived, and the necessary and sufficient conditions for the existence of the strategy are presented. Then the Pareto optimal solution under the worst-case disturbance is solved. Finally, the efficiency of the obtained results is illustrated by a numerical example.

Guaranteed Safe Spacecraft Docking With Control Barrier Functions

This paper presents a strategy for control of a spacecraft docking with a non-maneuvering target in the presence of safety constraints and bounded disturbances. The presence of disturbances prevents convergence to a unique docking state, so in our formulation, docking is defined as occurring within a set constructed using prescribed tolerances. Safety is ensured via application of Robust Control Barrier Functions to render a designated safe set forward invariant for any allowable disturbance. However, this safety strategy necessarily presumes a worst-case disturbance, and thus restricts trajectories to a subset of the safe set when a worst-case disturbance is not present. The presented controller accounts for this restriction, and guarantees that the spacecraft both remains safe and achieves docking in finite time for any allowable disturbance. The controller is then validated in simulation for a spacecraft landing on an asteroid, and two spacecraft docking in low Earth orbit.

Stagewise Newton Method for Dynamic Game Control With Imperfect State Observation

In this letter, we study dynamic game optimal control with imperfect state observations and introduce an iterative method to find a local Nash equilibrium. The algorithm consists of an iterative procedure combining a backward recursion similar to minimax differential dynamic programming and a forward recursion resembling a risksensitive Kalman smoother. A coupling equation renders the resulting control dependent on the estimation. In the end, the algorithm is equivalent to a Newton step but has linear complexity in the time horizon length. Furthermore, a merit function and a line search procedure are introduced to guarantee convergence of the iterative scheme. The resulting controller reasons about uncertainty by planning for the worst case disturbances. Lastly, the low computational cost of the proposed algorithm makes it a promising method to do output-feedback model predictive control on complex systems at high frequency. Numerical simulations on realistic robotic problems illustrate the risk-sensitive behavior of the resulting controller.

Integral reinforcement learning-based guaranteed cost control for unknown nonlinear systems subject to input constraints and uncertainties

This paper investigates guaranteed cost control (GCC) problem for nonlinear systems subject to input constraints and disturbances by utilizing the reinforcement-learning (RL) algorithm. Firstly, by establishing a modified Hamilton–Jacobi–Bellman (HJI) equation, which is difficult to be solved, a model-based policy iteration (PI) GCC algorithm is designed for input-constrained nonlinear systems with disturbances. Moreover, without requiring any knowledge of system dynamics, by designing an auxiliary system with a control law and an auxiliary disturbance policy, an online model-free GCC approach is developed by utilizing integral reinforcement learning (IRL) algorithm. To implement the proposed control algorithm, the actor and disturbance NNs are constructed to approximate the optimal control input and worst-case disturbance policy, while the critic NN is utilized to approximate optimal value function. Further, a synchronization weight update law is developed to minimize the NN approximation residual errors. The asymptotic stability of controlled systems is analyzed by applying the Lyapunov’s method. Finally, the effectiveness and feasibility of the proposed control method are verified by two nonlinear simulation examples.

Optimal control and learning for cyber‐physical systems

Optimal control and learning for <scp>cyber‐physical</scp> systems

Nonlinear input/output analysis: application to boundary layer transition

Abstract