Control Of Dynamic Systems Research Articles

Sliding Mode Control in Dynamic Systems

Due to its inherent robustness and finite time convergence, sliding mode control (SMC) is extensively used for the control of nonlinear uncertain systems [...]

Optimization-Based Design and Control of Dynamic Systems

Dynamic systems such as robots, autonomous vehicles, and process plants require careful design and control to achieve optimal performance. This paper presents an integrated framework for the simultaneous optimization of system design parameters and control policies. The underlying mathematical formulation utilizes optimization techniques to find the best system configuration and control strategy according to specified objectives. These objectives may include metrics such as tracking error, energy consumption, safety margins, and cost. Both model-based and data-driven techniques are explored for learning the system dynamics and synthesizing optimal controllers. The optimization algorithms leveraged include gradient-based methods, evolutionary algorithms, and reinforcement learning. The benefits of the joint optimization approach are demonstrated through case studies on representative dynamic systems. Results show that the integrated design and control framework outperforms sequential optimization, leading to improved efficiency, responsiveness, and robustness. This underscores the importance of co-optimizing design and control parameters, especially for complex, uncertain systems. The proposed methods provide an effective tool for next-generation automated design of smart, adaptable systems.

Learning Control Policies for Stochastic Systems with Reach-Avoid Guarantees

We study the problem of learning controllers for discrete-time non-linear stochastic dynamical systems with formal reach-avoid guarantees. This work presents the first method for providing formal reach-avoid guarantees, which combine and generalize stability and safety guarantees, with a tolerable probability threshold p in [0,1] over the infinite time horizon. Our method leverages advances in machine learning literature and it represents formal certificates as neural networks. In particular, we learn a certificate in the form of a reach-avoid supermartingale (RASM), a novel notion that we introduce in this work. Our RASMs provide reachability and avoidance guarantees by imposing constraints on what can be viewed as a stochastic extension of level sets of Lyapunov functions for deterministic systems. Our approach solves several important problems -- it can be used to learn a control policy from scratch, to verify a reach-avoid specification for a fixed control policy, or to fine-tune a pre-trained policy if it does not satisfy the reach-avoid specification. We validate our approach on 3 stochastic non-linear reinforcement learning tasks.

STL-Based Synthesis of Feedback Controllers Using Reinforcement Learning

Deep Reinforcement Learning (DRL) has the potential to be used for synthesizing feedback controllers (agents) for various complex systems with unknown dynamics. These systems are expected to satisfy diverse safety and liveness properties best captured using temporal logic. In RL, the reward function plays a crucial role in specifying the desired behaviour of these agents. However, the problem of designing the reward function for an RL agent to satisfy complex temporal logic specifications has received limited attention in the literature. To address this, we provide a systematic way of generating rewards in real-time by using the quantitative semantics of Signal Temporal Logic (STL), a widely used temporal logic to specify the behaviour of cyber-physical systems. We propose a new quantitative semantics for STL having several desirable properties, making it suitable for reward generation. We evaluate our STL-based reinforcement learning mechanism on several complex continuous control benchmarks and compare our STL semantics with those available in the literature in terms of their efficacy in synthesizing the controller agent. Experimental results establish our new semantics to be the most suitable for synthesizing feedback controllers for complex continuous dynamical systems through reinforcement learning.

State constrained stochastic optimal control for continuous and hybrid dynamical systems using DFBSDE

We develop a computationally efficient learning-based forward–backward stochastic differential equations (FBSDE) controller for both continuous and hybrid dynamical (HD) systems subject to stochastic noise and state constraints. Solutions to stochastic optimal control (SOC) problems satisfy the Hamilton–Jacobi–Bellman (HJB) equation. Using current FBSDE-based solutions, the optimal control can be obtained from the HJB equations using deep neural networks (e.g., long short-term memory (LSTM) networks). To ensure the learned controller respects the constraint boundaries, we enforce the state constraints using a soft penalty function. In addition to previous works, we adapt the deep FBSDE (DFBSDE) control framework to handle HD systems consisting of continuous dynamics and a deterministic discrete state change. We demonstrate our proposed algorithm in simulation on a continuous nonlinear system (cart–pole) and a hybrid nonlinear system (five-link biped).

Detecting Stability and Bifurcation Points in Control-Based Continuation for a Physical Experiment of the Zeeman Catastrophe Machine

Feedback-based control of dynamical systems enables tracking of unstable branches that cannot be reached by the free evolution of a simulated or physical system. As the stationary points are stabilized with these control methods, the stability information on the original system gets lost in the data from the transient behavior reaching these equilibria. We introduce, by means of a physical experiment of the Zeeman catastrophe machine, a method to construct the whole bifurcation diagram including stability information about the underlying uncontrolled equilibria and the location of bifurcation points.

Physics-informed neural ODE (PINODE): embedding physics into models using collocation points

Building reduced-order models (ROMs) is essential for efficient forecasting and control of complex dynamical systems. Recently, autoencoder-based methods for building such models have gained significant traction, but their demand for data limits their use when the data is scarce and expensive. We propose aiding a model’s training with the knowledge of physics using a collocation-based physics-informed loss term. Our innovation builds on ideas from classical collocation methods of numerical analysis to embed knowledge from a known equation into the latent-space dynamics of a ROM. We show that the addition of our physics-informed loss allows for exceptional data supply strategies that improves the performance of ROMs in data-scarce settings, where training high-quality data-driven models is impossible. Namely, for a problem of modeling a high-dimensional nonlinear PDE, our experiments show times 5 performance gains, measured by prediction error, in a low-data regime, times 10 performance gains in tasks of high-noise learning, times 100 gains in the efficiency of utilizing the latent-space dimension, and times 200 gains in tasks of far-out out-of-distribution forecasting relative to purely data-driven models. These improvements pave the way for broader adoption of network-based physics-informed ROMs in compressive sensing and control applications.

Control simulation experiments of extreme events with the Lorenz-96 model

Abstract. The control simulation experiment (CSE) is a recently developed approach to investigate the controllability of dynamical systems, extending the well-known observing system simulation experiment (OSSE) in meteorology. For effective control of chaotic dynamical systems, it is essential to exploit the high sensitivity to initial conditions for dragging a system away from an undesired regime by applying minimal perturbations. In this study, we design a CSE for reducing the number of extreme events in the Lorenz-96 model. The 40 variables of this model represent idealized meteorological quantities evenly distributed on a latitude circle. The reduction of occurrence of extreme events over 100-year runs of the model is discussed as a function of the parameters of the CSE: the ensemble forecast length for detecting extreme events in advance, the magnitude and localization of the perturbations, and the quality and coverage of the observations. The design of the CSE is aimed at reducing weather extremes when applied to more realistic weather prediction models.

MULTI-MOTOR DRIVE NON-PARAMETRIC BLACK-BOX FUZZY MODEL

The multi-motor drives, the typical examples of which are continuous lines (CL) for tension processing of various materials are complex and coupled MIMO higher-order nonlinear systems, which parameters are difficult to identify. The article focuses on a method for the design of a non-parametric black-box fuzzy model of a continuous production line with emphasis on minimal knowledge on the modelled system. The proposed fuzzy model structure is based on the state space representation of the dynamic system in discrete form, which only requires a suitable set of information on its input/output relations. The properties of the proposed CL fuzzy model were verified by experimental measurements on a CL laboratory model. The obtained results have confirmed the rightness and effectiveness of the fuzzy model design method that can be applied not only in the field of industry technologies with CL but also in modeling and control of nonlinear dynamic systems.

CACTO: Continuous Actor-Critic With Trajectory Optimization—Towards Global Optimality

This paper presents a novel algorithm for the continuous control of dynamical systems that combines Trajectory Optimization (TO) and Reinforcement Learning (RL) in a single framework. The motivations behind this algorithm are the two main limitations of TO and RL when applied to continuous nonlinear systems to minimize a non-convex cost function. Specifically, TO can get stuck in poor local minima when the search is not initialized close to a “good” minimum. On the other hand, when dealing with continuous state and control spaces, the RL training process may be excessively long and strongly dependent on the exploration strategy. Thus, our algorithm learns a “good” control policy via TO-guided RL policy search that, when used as initial guess provider for TO, makes the trajectory optimization process less prone to converge to poor local optima. Our method is validated on several reaching problems featuring non-convex obstacle avoidance with different dynamical systems, including a car model with 6D state, and a 3-joint planar manipulator. Our results show the great capabilities of CACTO in escaping local minima, while being more computationally efficient than the Deep Deterministic Policy Gradient (DDPG) and Proximal Policy Optimization (PPO) RL algorithms.

Design and development of brain emotional learning based adaptive membership functions for type-2 fuzzy systems

This paper presents the design and development of Brain Emotional Learning based adaptive Type-2 Fuzzy Systems for control of dynamical systems. The BEL controller belongs to the class of bio inspired controllers, as its architecture is based on limbic system of human brain and is capable of providing solutions for complex real time problems. In this work, dynamics of Brain Emotional Learning are used for the adaptation of membership functions in the design of Type-2 Fuzzy Logic Controllers. The stability of the overall system is analysed through Lyapunov Yakubovich’s criteria. The proposed approach is validated on the benchmark system such as inverted pendulum, CSTR and Ship heading control through simulation and in real-time environment using OPAL RT OP5600.

Robust adaptive control of nonlinear dynamic systems using hybrid sliding mode regressive neural learning technique

Purpose The proposed Sliding Mode Control-Global Regressive Neural Network (SMC-GRNN) algorithm is an integration of Global Regressive Neural Network (GRNN) and Sliding Mode Control (SMC). Through this integration, a novel structure of GRNN is designed to enable online and. This structure is then combined with SMC to develop a stable adaptive controller for a class of nonlinear multivariable uncertain dynamic systems.Design/methodology/approach In this study, a new hybrid (SMC-GRNN) control method is innovatively developed.Findings A novel structure of GRNN is designed that can be learned online and then be integrated with the SMC to develop a stable adaptive controller for a class of nonlinear uncertain systems. Furthermore, Lyapunov stability theory is utilized to ensure the hidden-output weighting values of SMC-GRNN adaptively updated in order to guarantee the stability of the closed-loop dynamic system. Eventually, two different numerical benchmark tests are employed to demonstrate the performance of the proposed controller.Originality/value A novel structure of GRNN is originally designed that can be learned online and then be integrated with the sliding mode SMC control to develop a stable adaptive controller for a class of nonlinear uncertain systems. Moreover, Lyapunov stability theory is innovatively utilized to ensure the hidden-output weighting values of SMC-GRNN adaptively updated in order to guarantee the stability of the closed-loop dynamic system.

Adaptive path-following control for parafoil dynamic systems with wind disturbance and rate constraint

Adaptive path-following control for parafoil dynamic systems with wind disturbance and rate constraint

Теоретичне дослідження стійкості руху асиметричного посівного машинно-тракторного агрегату

Goal. Theoretical evaluation of the effect of asymmetric connection of the row drill to the tractor on the stability of the movement of the sowing unit in the horizontal plane. Methods. Theoretical studies were carried out using the methods of theoretical mechanics, higher mathematics and the theory of automatic control of dynamic systems. Numerical calculations and construction of graphical dependencies were performed on a PC using the Mathcad 15.0 programming environment. Results. Practically all existing machine-tractor units for sowing row crops (corn, sunflower, soybean, etc.) are symmetrical. This is because each row seeder has an even number of sowing sections, and the ratio of the size of the tractor track to the row width of the row crop is an even integer (k). This article presents the results of research on the sowing unit, in which, with an even number of sowing sections of the seeder, the number k is equal to 3, which is odd. As a result, an asymmetric unit was obtained, in which the seeder is offset in the transverse direction from the axis of symmetry of the tractor by half the width of the row spacing, which is equal to 70 cm. In practice, such displacement of the seeder is carried out by hanging it on the tractor with the help of a specially designed transition device. Conclusions. With the use of mathematical modeling of the movement of the seeding machine-tractor unit in the horizontal plane, it is shown that the right-side transverse displacement of the planter by 0.35 m forms a constantly acting turning moment. But it has less effect on the stability of the movement of the sowing unit than the similar moment created by the marker. Calculations have established that increasing the input resistance coefficients of the tires of the tractor wheels, which can be achieved by correspondingly increasing the air pressure in them, leads to a desired decrease in the amplitude-frequency characteristics of the seeding unit’s development of the turning disturbance moment. Such a solution is more effective concerning the tires of the tractor’s rear wheels.

Data-driven control for dynamic quantized nonlinear systems with state constraints based on barrier functions

In this study, the data-driven control problem is investigated for discrete-time nonlinear systems with state constraints and dynamic quantization. A dynamic quantizer is utilized to encode the control signal to relieve the impact of inherent quantization errors in comparison with traditional uniform quantizers. By employing three sets of neural networks (NNs), a data-based goal representation heuristic dynamic programming algorithm with the concurrent learning technique is offered to achieve the near-optimal control objective without model requirement. Specifically, by introducing the relaxed barrier function into the cost function, a barrier-based control strategy is proposed to generate the repulsive control force when the state moves to the constraint boundary. Subsequently, the recorded historical data is adequately leveraged to build a new weight updating rule, and persistent excitation of weight parameter estimation is hence removed. Furthermore, the Lyapunov method is utilized to demonstrate that the weights estimation errors of NNs are bounded. Finally, numerical simulation and power system examples are employed to demonstrate the validity of the control strategy.

Estimation and stochastic control of nonlinear dynamic systems over the AWGN channel: Application in tele‐presence and tele‐operation of autonomous vehicles

AbstractThis paper is concerned with tele‐presence and tele‐operation of autonomous vehicles over the Additive White Gaussian Noise (AWGN) channel. This wireless communication channel is subject to transmission noise and transmission power constraint. An encoder, decoder and controller are proposed by implementing a novel linearization method for linearizing the nonlinear dynamic systems at operating points. For the linearized systems, the encoder, decoder and controller are implemented that were previously proposed for output tracking as well as stability of linear dynamic systems. Two novel linearization methods are proposed and their satisfactory performances are proved for output tracking as well as stability of nonlinear dynamic system. The first method is based on the fixed linearization rate and the second one is based on the variable (optimal) linearization rate. The decoder that is based on the second method is in fact the extended Kalman filter with the optimal linearization rate. The performances of these two methods are compared with each other and the other proposed methods by implementing them to the problems of the tele‐presence and tele‐operation of autonomous vehicles over the AWGN channel. Their satisfactory performances are illustrated. It is illustrated that the second method has higher computational complexity with slightly better performance and is applicable to a larger class of reference trajectories.

Finite-Time Adaptive Fuzzy Control for Unmodeled Dynamical Systems with Actuator Faults

This article concentrates upon the issue of finite-time tracking control for a category of nonlinear systems in pure-feedback form with actuator faults and unmodeled dynamics, where the loss of effectiveness and bias fault are considered. Meanwhile, the function approximation method utilizing fuzzy logic systems and dynamic surface control approach with first-order filter are implemented to model the unknown nonlinear terms induced from the proposed controller procedure and tackle the “explosion of complexity” issue of the classic backstepping method. The use of the maximal norm of the weight vector estimation method and adaptive approach reduces the computation load induced by fuzzy logic systems. Within the framework of backstepping control, a finite-time adaptive fuzzy fault-tolerant control protocol is derived to guarantee the boundedness of all signals and tracking error of the controlled system within a finite-time. Simulation studies are offered to show the validity of the derived theoretical results of the finite-time control protocol.

Online Reinforcement Learning Control of Nonlinear Dynamic Systems: A State-action Value Function Based Solution

In this paper, we present an online reinforcement learning-based solution to the optimal control problem of continuous-time nonlinear input-affine systems. The proposed approach contains a concurrent identifier that estimates time derivatives of states of the system in some arbitrary points. The identifier is utilized to simulate a so-called Bellman error in some unvisited points. The simulated errors together with errors obtained along the trajectory of the system are used to estimate the state-action value function, which is then employed to derive the estimated optimal controller. The designed approach does not explicitly require the input dynamics, which is hard to segregate it from the drift dynamics in optimal regulation problems. In addition, the simulated Bellman errors relax the confining persistence of excitation condition, which is needed for convergence in deterministic systems. A Lyapunov-based analysis was conducted to derive convergence conditions. Simulation studies demonstrated the effectiveness of the developed control scheme.

Control of chaos with time-delayed feedback based on deep reinforcement learning

The time-delayed feedback control method, as one of popular methods for chaos control, is noninvasive and flexible for various dynamical systems from different fields of science and technology. In the method, however, an appropriate choice of the feedback gain is challenging, which requires the explicit mathematical model of the controlled system for stability analysis. Additionally, another limitation of the method is the so-called odd number limitation. The two problems restrict its application to practical situations. Fortunately, a technique called deep reinforcement learning is capable of learning the controlled environment (the dynamical system), by continually interacting with the controlled system, which makes it possible to obtain the right feedback gain without the requirement of the accurate mathematical model of the system. Hence, in this work, a time-delayed feedback control method based on deep reinforcement learning is put forward to solve the two problems. Compared with the traditional time-delayed feedback control, the proposed method, as a data-driven method due to the combination with deep reinforcement learning, offers a time-varying feedback gain according to the well-trained policy learned by the deep reinforcement learning algorithm. It maintains the non-invasive property of the time-delayed feedback control, but expands the operating range due to the time-varying feedback gain overcoming the odd number limitation. With numerical simulations, the proposed method is successfully applied to three different kinds of systems, the discrete logistic map, the non-autonomous Duffing oscillator and the autonomous Lorenz system.

Boundary Fuzzy Output Tracking Control of Nonlinear Parabolic Infinite-Dimensional Dynamic Systems: Application to Cooling Process in Hot Strip Mills

This paper utilizes a combination of integral control, fuzzy control, and observer-based output feedback control to deal with the issue of nonlinear output tracking control (OTC) design for nonlinear infinite-dimensional dynamic systems. The system dynamics model is represented by a semi-linear parabolic partial differential equation (PDE) with boundary control and non-collocated boundary measurement. Initially, a Takagi-Sugeno (T-S) fuzzy parabolic PDE model is constructed to surmount the OTC design difficulty from the infinite-dimensional nonlinear system dynamics. Subsequently, a fuzzy-observer-based OTC law is proposed via the T-S fuzzy PDE model and the integral control approach. Here, the integral control ensures asymptotic output regulation, and the observer-based output feedback control is employed to conquer the stabilizing control design difficulty caused by the non-collocation between control actuation and measurement. It is shown via the Lyapunov technique with variants of vector-valued Poincaré-Wirtinger's inequality that the suggested fuzzy OTC law drives the measurement output to asymptotically track the desired reference signal and ensures the boundedness of the resulting closed-loop system, provided that a sufficient condition given in the form of linear matrix inequalities is fulfilled. Moreover, the proposed fuzzy-model-based OTC design is also revised for the exponential stabilization case. Finally, extensive simulation results for a numerical example and a cooling process in hot strip mills are provided to examine the effectiveness and merit of the proposed fuzzy OTC scheme.