Continuous-time Control Research Articles

From NeurODEs to AutoencODEs: A mean-field control framework for width-varying neural networks

Abstract The connection between Residual Neural Networks (ResNets) and continuous-time control systems (known as NeurODEs) has led to a mathematical analysis of neural networks, which has provided interesting results of both theoretical and practical significance. However, by construction, NeurODEs have been limited to describing constant-width layers, making them unsuitable for modelling deep learning architectures with layers of variable width. In this paper, we propose a continuous-time Autoencoder, which we call AutoencODE, based on a modification of the controlled field that drives the dynamics. This adaptation enables the extension of the mean-field control framework originally devised for conventional NeurODEs. In this setting, we tackle the case of low Tikhonov regularisation, resulting in potentially non-convex cost landscapes. While the global results obtained for high Tikhonov regularisation may not hold globally, we show that many of them can be recovered in regions where the loss function is locally convex. Inspired by our theoretical findings, we develop a training method tailored to this specific type of Autoencoders with residual connections, and we validate our approach through numerical experiments conducted on various examples.

Observer-based event-triggered boundary control of the one-phase Stefan problem

This paper presents an observer-based event-triggered boundary control strategy for the one-phase Stefan problem, utilising the position and velocity measurements of the moving interface. The design of the observer and controller is founded on the infinite-dimensional backstepping approach. To implement the continuous-time observer-based controller in an event-triggered framework, we propose a dynamic event triggering condition. This condition specifies the instances when the control input must be updated. Between events, the control input is maintained constant in a Zero-Order-Hold manner. We demonstrate that the dwell-time between successive triggering moments is uniformly bounded from below, thereby precluding Zeno behaviour. The proposed event-triggered boundary control strategy ensures the well-posedness of the closed-loop system and the satisfaction of certain model validity conditions. Additionally, the global exponential convergence of the closed-loop system to the setpoint is established using Lyapunov approach. A simulation example is provided to validate the theoretical findings.

Optimal Output Consensus of Heterogeneous Linear Multiagent Systems Over Weight-Unbalanced Directed Networks.

This article investigates the distributed optimal output consensus problem of heterogeneous linear multiagent systems over weight-unbalanced directed networks. A novel distributed continuous-time state feedback controller is proposed to steer the outputs of all the agents to converge to the optimal solution of the global cost function. Under the standard condition that the unbalanced digraph is strongly connected and the local cost functions are strongly convex with global Lipschitz gradients, the exponential convergence of the closed-loop multiagent system is established. Then, the proposed state feedback control law is extended to an observer-based output feedback setting. Two examples are finally provided to illustrate the effectiveness of the proposed control schemes.

Euler implicit time-discretization of multivariable sliding-mode controllers

This paper aims to develop the Euler implicit time-discretization of multivariable sliding-mode controllers to solve the numerical chattering problem without modifying the continuous-time control law. To this end, a continuous-time multi-input plant under a multivariable sliding-mode control is studied, and it is shown that the implicit discretization of the continuous-time sliding-mode controller leads to a multivariable generalized equation with several set-valued terms which is not possible to be solved using the graphical interpretations. Subsequently, an algorithm is proposed to solve such a multivariable generalized equation required to synthesize the implicit sliding-mode control signal at each time step. The proposed algorithm is explained through a simple example accompanied by numerical simulations. The properties of the implicit multivariable sliding-mode controller, e.g., finite-time convergence, gain insensitivity, and chattering suppression, are studied analytically. Afterwards, a multivariable sliding-mode controller is implemented on a digital processor based on the developed algorithm to control a six-input six-output system, i.e., six-component thrust generator, and the results are compared with the case where the continuous-time sliding-mode controller is implemented using the conventional Euler explicit discretization. In the end, the related issues and drawbacks are addressed to be considered in future works.

Value iteration algorithm for continuous-time linear quadratic stochastic optimal control problems

Value iteration algorithm for continuous-time linear quadratic stochastic optimal control problems

High-Gain-Observer-Based Predictive Output Feedback for Nonlinear Systems With Large Input-Delays

Abstract The predictor feedback has been demonstrated to be quite effective in large delay compensation. However, few researches in the field of predictor feedback for large delays focused on output feedback control (OFC). This paper develops the previous work to design high-gain-observer-based predictive output feedback for nonlinear systems with large delays. Two methods are employed for large delay compensation: the backstepping-based partial differential equation (PDE) method and the reduction-based ordinary differential equation (ODE) method. It appears that, for continuous-time control, the first method leads to simpler linear matrix inequality (LMI) conditions and deal with larger delays, whereas the second method is easily exploited for sampled-data implementation under continuous-time measurement. Lyapunov–Krasovskii method is presented to guarantee the exponential stability of the nonlinear systems under predictor-based controllers. Through a simulation example of pendulum, the proposed methods are demonstrated to be efficient when the input delays are too large for the system to be stabilized without a predictor.

Stability of Progressively Tightening Model Predictive Control in Continuous Time

We consider a continuous-time nonlinear model predictive control formulation that is progressively tightening in path costs and constraints. Under standard assumptions, we prove asymptotic stability of the origin for the corresponding closed-loop system and extend this result to formulations employing an auxiliary dynamic system. The theoretical results are illustrated on a numerical example.

Performance-Barrier-Based Event-Triggered Boundary Control of Congested ARZ Traffic PDEs

Employing the recent concept of performance-barrier-based event-triggered control (P-ETC), this study presents an event-triggered PDE backstepping design for stabilizing stop-and-go oscillations in Traffic flow by actuating a variable speed limit (VSL) located at the downstream boundary of a freeway segment. The system is described by the linearized Aw-Rascle-Zhang (ARZ) macroscopic Traffic model, a 2 × 2 coupled hyperbolic system. Compared to continuous-time control, event-triggered control (ETC) generates piecewise-constant variable speed limit (VSL) commands, which are more likely to be adhered to by human drivers. Unlike the existing regular ETC (R-ETC) approach that conservatively enforces strict decreases in the closed-loop system Lyapunov function, our proposed P-ETC approach allows for increases as long as the Lyapunov function remains below an acceptable performance barrier. This flexibility results in fewer control updates of P-ETC compared with R-ETC while ensuring Zeno-free behavior and exponential convergence to zero in the spatial L2 norm. Comparative simulation results illustrate the benefits of the performance-barrier design in terms of safety, driver comfort, travel time, fuel consumption. The proposed P-ETC reduces discomfort nearly in half relative to driver behavior without VSL, while increasing the driver safety by as much as 10×, measured by the average dwell time, relative to the R-ETC VSL schedule.

Discrete-time Control by Interconnection using energy-preserving collocation methods

The energy-preserving collocation methods combine a high order of accuracy with an exact discrete-time energy balance equation when discretizing port-Hamiltonian systems. We show how they can be used to translate Control by Interconnection to the discrete-time setting. We set up the discrete-time plant and controller model and compute the discrete-time inputs for which the desired equilibrium of the continuous-time closed-loop system becomes a stable equilibrium of the discrete-time models. To obtain a continuous-time control signal for the plant system in the sampling interval, these discrete inputs are then interpolated by a polynomial. The advantages of the proposed approach at large sampling times are illustrated in simulations using the example of the controlled pendulum.

Continuous-Time Control Synthesis Under Nested Signal Temporal Logic Specifications

Continuous-Time Control Synthesis Under Nested Signal Temporal Logic Specifications

Innovative Optimal Control Path Planning for PMCU in Cruise Ship Construction: A Kinematic and Obstacle Avoidance Model

In cruise ship manufacturing, it is common practice for prefabricated modular cabin units (PMCU) to be assembled in the workshop and transported to a designated installation location on board the ship. This paper proposed a Radau pseudospectral method based on a probabilistic roadmap (PRM) for PMCU path planning in complex deck environments. Firstly, a kinematic model is constructed, and the continuous-time optimal control path planning problem is established. Among them, an area-based obstacle avoidance constraint is applied to the approach. Then, the continuous-time optimal control problem is discretized into a nonlinear programming (NLP) problem using the orthogonal collocation method. For the initial value-sensitive NLP problem solving, a strategy using PRM combined with a heuristic rule is used to obtain a high-quality initial guess. Finally, experiments under two operating scenarios of PMCU pushing demonstrate that the method is capable of generating a better path while ensuring that the whole process is collision-free. The results show that the algorithm proposed in this paper can effectively solve the PMCU pushing path planning problem in the complex deck environment, and the path considering kinematic characteristics can be used for cabin installation, which has strong operability and applicability.

Closed-loop optimisation of neural networks for the design of feedback policies under uncertainty

Solving model predictive control (MPC) problems online can be computationally intractable, especially when considering uncertainty and nonlinear systems. One approach to avoid this is to train a neural network on a data-set of solutions of MPC problems (potentially nonlinear) offline, and to evaluate the trained control policy online. However, due to the separation of these optimisation problems the neural network controller must be carefully verified to ensure adequate closed loop performance. In this paper we propose a method to train a neural network in closed loop for control systems, in continuous or discrete time, while allowing for flexible consideration of parametric uncertainty. This method does not require off-line solutions of the NMPC problem, and instead directly optimises the desired closed loop performance. We prove that our method can approximate the optimal closed-loop control policy to arbitrary tolerance and in numerical examples demonstrate its performance compared to explicit MPC, imitation learning and nominal MPC.

Linear Convergence of a Policy Gradient Method for Some Finite Horizon Continuous Time Control Problems

Linear Convergence of a Policy Gradient Method for Some Finite Horizon Continuous Time Control Problems

Off‐policy model‐based end‐to‐end safe reinforcement learning

AbstractSafety and stability considerations play a crucial role in the development of learning based strategies for control design of systems that require high levels of safety. Safe reinforcement learning (RL) based approaches traditionally seek learning of the control laws that are optimal with respect to system performance whilst ensuring system stability and safety. In this article, an off‐policy safe RL based approach is proposed for nonlinear systems affine in control in continuous time. In this novel work, safety and stability are guaranteed during initialization and exploration phases by adjusting the control input with the solution of a quadratic programming problem combining both input to state stable‐control Lyapunov function and robust control barrier function (R‐CBF) conditions. Moreover, the safety of the learned policy is assured by augmenting the cost function with a CBF to maintain safety and optimize performance simultaneously. Novel mathematically rigorous proofs are provided to establish the stability and safety guarantees, offering a sound theoretical foundation for the approach. To demonstrate the effectiveness of the algorithm, two examples are presented: engine surge and stall dynamics, and an unstable nonlinear system.

Online continuous-time adaptive predictive control of the technological glass conditioning process

Online continuous-time adaptive predictive control of the technological glass conditioning process

Theoretical Foundations and Implications of Neural Ordinary Differential Equations (Nodes) For Real-Time Portfolio Optimization

This paper embarks on a comprehensive exploration of the theoretical landscape surrounding the integration of Neural Ordinary Differential Equations (NODEs) into the domain of real-time portfolio optimization. The study commences by establishing the background and motivation for this research, shedding light on the challenges encountered in real-time portfolio management and the potential transformative role NODEs can play in addressing these challenges. The theoretical framework unfolds in a structured manner, encompassing critical facets of portfolio optimization theory. It delves into classical portfolio optimization methodologies, including the mean- variance framework and continuous-time stochastic control techniques. This solid theoretical foundation provides the basis for understanding the nuances of optimizing portfolio weights, expected returns, and risk measures. The heart of the research lies in the integration of NODEs, an innovative fusion of deep learning and differential equations, into the fabric of portfolio optimization. NODEs, with their adaptability and ability to model continuous- time dynamics, emerge as a potent tool for real-time portfolio rebalancing and decision-making. The study provides an in-depth overview of NODEs, elucidating their architecture and their application in modeling financial time series data. This theoretical journey leads to the exploration of practical implications. The study highlights the potential benefits of incorporating NODEs into portfolio management, including improved risk management, enhanced returns, and the capacity for adaptive asset allocation strategies. However, it also addresses the limitations and challenges associated with this integration, such as data quality issues and computational requirements. In conclusion, this research presents a theoretical framework that bridges the gap between deep learning and continuous-time financial models, offering a promising avenue for real-time portfolio optimization. The insights derived from this study serve as a foundation for future research and practical applications in navigating the intricate landscape of financial markets.

Adaptive output feedback reinforcement learning control for continuous time switched stochastic nonlinear systems with unknown control coefficients and full-state constraints

In this paper, the problem of adaptive neural network (NN) reinforcement learning (RL) tracking control is investigated for the continuous time (CT) switched stochastic nonlinear systems with unknown control coefficients and full-state constraints. First, a set of reconstructed states are defined to handle the unknown control coefficients, and switched state observers are developed to estimate unmeasurable reconstructed states. Then, to improve the tracking performance, based on the minimal learning parameter (MLP) method and the RL control design technique, the adaptive RL controller is developed by the backstepping method. Finally, the boundedness of the tracking error and all signals is demonstrated via the average dwell time (ADT) method and tangent type time-varying barrier multiple Lyapunov functions. The effectiveness of the proposed scheme is verified by two examples.

Analysis of Theoretical and Numerical Properties of Sequential Convex Programming for Continuous-Time Optimal Control

Sequential Convex Programming (SCP) has recently gained significant popularity as an effective method for solving optimal control problems and has been successfully applied in several different domains. However, the theoretical analysis of SCP has received comparatively limited attention, and it is often restricted to discrete-time formulations. In this paper, we present a unifying theoretical analysis of a fairly general class of SCP procedures for continuous-time optimal control problems. In addition to the derivation of convergence guarantees in a continuous-time setting, our analysis reveals two new numerical and practical insights. First, we show how one can more easily account for manifold-type constraints, which are a defining feature of optimal control of mechanical systems. Second, we show how our theoretical analysis can be leveraged to accelerate SCP-based optimal control methods by infusing techniques from indirect optimal control.

Abstraction of Continuous-Time Systems Based on Feedback Controllers and Mixed Monotonicity

In this paper, we consider the problem of the computation of efficient symbolic abstractions for continuous-time control systems. The new abstraction algorithm builds symbolic models with the same number of states but fewer transitions in comparison to the one produced by the standard algorithm. At the same time, the new abstract system is at least as controllable as the standard one. The proposed algorithm is based on the solution of a region-to-region control synthesis problem. This solution is formally obtained using the theory of viscosity solutions of the dynamic programming equation and the theory of differential equations with discontinuous righthand side. In the new abstraction algorithm, the symbolic controls are essentially the feedback controllers that solve this control synthesis problem. The improvement in the number of transitions is achieved by reducing the number of successors for each symbolic control. For a certain class of control systems, with a suitable set of discretization parameters, the new algorithm may even produce deterministic abstract systems or systems with a singleton input alphabet. The approach is illustrated by examples that compare the two abstraction algorithms.

Event-Triggered Control for a Second Order ODE-Heat System Coupling at Intermediate Point

Motivated by the thermoelastic coupling effect arising in microelectromechanical systems (MEMS), event-triggered control of second-order ODE-heat systems coupled at intermediate point is investigated. The event-triggered control includes two parts, one is the feedback control signal, and the other is the event-triggered mechanism that decides when to update the control input. First, we design an event-triggered controller by using Zero-Order Hold to a continuous-time controller. Then, a dynamic triggering condition is established. Next, we derive that the existence of a minimal dwell-time, which avoids the occurrence of Zeno behavior. By utilizing Lyapunov-Krasovskii functional method, the global exponential stability of the closed-loop system is demonstrated. Finally, the simulation data based on the thermoelastic coupling system is displayed to demonstrate the availability of the theoretical results.