Ensure Constraint Satisfaction Research Articles

Robust preventive and corrective security-constrained OPF for worst contingencies with the adoption of VPP: A safe reinforcement learning approach

The rising frequency of extreme weather events calls for urgent measures to improve the resilience and reliability of power systems. This paper, therefore, presents a robust preventive-corrective security-constrained optimal power flow (PCSCOPF) model designed to strengthen power system reliability during N-k outages. The model integrates fast-response virtual power plants (VPPs), dynamically adjusting their injections to mitigate post-contingency overloads and maintain branch flows within emergency limits. Additionally, a novel approach combining deep reinforcement learning (DRL) with Lagrangian relaxation is introduced to efficiently solve the PCSCOPF decision-making problem. By framing the problem as a constrained Markov decision process (CMDP), the proposed Lagrangian-based soft actor-critic (L-SAC) algorithm optimizes control actions while ensuring constraint satisfaction during the exploration process. Extensive investigations have been conducted on the IEEE 30-bus and 118-bus systems to evaluate their computational efficiency and reliability.

Modular control architecture for safe marine navigation: Reinforcement learning with predictive safety filters

Many autonomous systems are safety-critical, making it essential to have a closed-loop control system that satisfies constraints arising from underlying physical limitations and safety aspects in a robust manner. However, this is often challenging to achieve for real-world systems. For example, autonomous ships at sea have nonlinear and uncertain dynamics and are subject to numerous time-varying environmental disturbances such as waves, currents, and wind. There is increasing interest in using machine learning-based approaches to adapt these systems to more complex scenarios, but there are few standard frameworks that guarantee the safety and stability of such systems. Recently, predictive safety filters (PSF) have emerged as a promising method to ensure constraint satisfaction in learning-based control, bypassing the need for explicit constraint handling in the learning algorithms themselves. The safety filter approach leads to a modular separation of the problem, allowing the use of arbitrary control policies in a task-agnostic way. The filter takes in a potentially unsafe control action from the main controller and solves an optimization problem to compute a minimal perturbation of the proposed action that adheres to both physical and safety constraints. In this work, we combine reinforcement learning (RL) with predictive safety filtering in the context of marine navigation and control. The RL agent is trained on path-following and safety adherence across a wide range of randomly generated environments, while the predictive safety filter continuously monitors the agents' proposed control actions and modifies them if necessary. The combined PSF/RL scheme is implemented on a simulated model of Cybership II, a miniature replica of a typical supply ship. Safety performance and learning rate are evaluated and compared with those of a standard, non-PSF, RL agent. It is demonstrated that the predictive safety filter is able to keep the vessel safe, while not prohibiting the learning rate and performance of the RL agent.

The chemostat reactor: A stability analysis and model predictive control

This contribution develops the model predictive control for an unstable chemostat reactor. Initially, we analyze the system’s model — a nonlinear first-order hyperbolic partial integro-differential equation (PIDE) — and carry the model linearization around an unstable operating condition. Employing the structure-preserving Cayley–Tustin transformation, we obtain a discrete-time model representation of the continuous model. Subsequently, we solve the operator Ricatti equations in the discrete-time setting to derive a full state feedback controller that stabilizes the closed-loop and design a Luenberger observer for state reconstruction given the system output measures. Finally, we formulate a dual-mode MPC ensuring constraint satisfaction and optimality, integrating the gain-based unconstrained full-state feedback optimal control obtained from the Ricatti equation. This dual-mode strategy describes an optimization problem where the predictive controller acts only if constraints become active within the control horizon. Simulation studies validate the controller performance, where the MPC only takes action if the constraints are predicted to be active within the control horizon while also guaranteeing closed-loop stabilization under only output feedback. This type of controller can be easily implemented with other control strategies and significantly decreases the computational costs of solving the optimal control problems when compared to other MPC approaches.

An Inexact-Penalty Method for GNE Seeking in Games With Dynamic Agents

We consider a network of autonomous agents whose outputs are actions in a game with coupled constraints. In such network scenarios, agents seek to minimize coupled cost functions using distributed information while satisfying the coupled constraints. Current methods consider the small class of multi-integrator agents using primal-dual methods. These methods can only ensure constraint satisfaction in steady-state. In contrast, we propose an inexact penalty method using a barrier function for nonlinear agents with equilibrium-independent passive dynamics. We show that these dynamics converge to an <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\varepsilon$</tex-math></inline-formula> -GNE while satisfying the constraints for all time, not only in steady-state. We develop these dynamics in both the full-information and partial-information settings. In the partial-information setting, dynamic estimates of the others' actions are used to make decisions and are updated through local communication. Applications to optical networks, velocity synchronization of flexible robots and wind farm optimization are provided.

A chance‐constrained tube‐based model predictive control for tracking linear systems using data‐driven uncertainty sets

AbstractThis article presents a chance‐constrained tube‐based model predictive control (MPC) method for tracking linear time‐invariant systems based on data‐driven uncertainty sets. By defining the terminal admissible set to consider all the possible steady‐states and reformulating the stochastic tube‐based MPC framework, the proposed method can systematically hedge against the impact of uncertainties and ensure tracking for all reachable operating setpoints. To reduce the conservatism of control performance while enlarging the feasible region, a data‐driven polyhedral uncertainty set is constructed by using the principal component analysis technique, which can effectively capture correlations among uncertain variables. Since state constraint violations in a certain probability are allowed, a probability uncertainty set is constructed by using statistic limit and cutting plane methods to formulate a stochastic tube to ensure constraint satisfaction. The recursive feasibility and stability can be guaranteed if the uncertainties are bounded. The effectiveness of the proposed method is verified by numerical examples and tracking problems of a thickening process.

Event-Triggered Adaptive Fuzzy Finite-Time Output Feedback Control for Stochastic Nonlinear Systems With Input and Output Constraints

This paper focuses on the problem of designing an adaptive fuzzy event-triggered finite-time output feedback control for stochastic nonlinear systems with input and output constraints. A fuzzy observer is designed to estimate the unmeasured states. The quartic asymmetric time-varying barrier Lyapunov function is established to ensure constraint satisfaction. By utilizing the stochastic theory, finite-time command filtered backstepping method and event-triggered mechanism, a finite-time event-triggered controller is recursively designed, which can not only guarantee finite-time convergent property, but also reduce communication pressure. Meanwhile, the matter of “explosion of complexity” is removed by introducing the finite-time command filter and the effect of filtered errors is offset by constructing error compensation signals. Moreover, an auxiliary system is introduced to handle the input constraint. Finally, the effectiveness of the theoretical results is demonstrated by the simulation example.

Constrained profile estimation for distributed parameter system in one dimension using orthogonal collocation

Several industrial processes fall under the category of Distributed parameter systems (DPS) which are represented as coupled partial differential equations (PDEs). Obtaining accurate and physically meaningful profile estimates of states despite only a limited number of spatially distributed measurements, is a pre-requisite for adequately controlling such systems. In this work, we propose an online constrained state profile estimation approach for DPS. The approach is based on reduced-dimension model represented by differential and algebraic equations (DAE) which in turn is developed by discretization of the PDEs along the spatial coordinate using the orthogonal collocation (OC) method. A profile prediction error based approach is proposed for systematically fixing the dimension of the OC model to be used. Further, to compensate for the inevitable model plant mismatch (MPM) arising in the process of the dimensionality reduction, additive zero mean Gaussian white noise processes are introduced in both state dynamics as well as measurement model. Statistics of these noises are obtained based on simulation based realizations of approximation errors at collocation points and measurement locations. To incorporate the identified correlation between the resulting state and measurement noises in the estimation method, we propose a constrained DAE-EKF (extended Kalman Filter) extension to deal with such correlations. The estimation approach also involves an appropriately formulated optimization based update step to ensure constraint satisfaction over the entire spatial profile leading to physically meaningful state profile estimates. The constrained profile estimator is also integrated with an online moving window based parameter estimation, which enables tracking of both time varying parameters and state profiles. The efficacy of the proposed approach is demonstrated by conducting stochastic simulation studies using two tubular reactor systems. The results show that the proposed constrained online state estimation approach is able to obtain accurate profile estimates at low computational cost, while ensuring that the profile estimates satisfy the given constraints.

Bayesian Optimization Framework for HVAC System Control

The use of machine-learning algorithms in optimizing the energy efficiency of HVAC systems has been widely studied in recent years. Previous research has focused mainly on data-driven model predictive controls and reinforcement learning. Both approaches require a large amount of online interactive data; therefore, they are not efficient and stable enough for large-scale practical applications. In this paper, a Bayesian optimization framework for HVAC control has been proposed to achieve near-optimal control performance while also maintaining high efficiency and stability, which would allow it to be implemented in a large number of projects to obtain large-scale benefits. The proposed framework includes the following: (1) a method for modeling HVAC control problems as contexture Bayesian optimization problems and a technology for automatically constructing Bayesian optimization samples, which are based on time series raw trending data; (2) a Gaussian process regression surrogate model for the objective function of optimization; (3) a Bayesian optimization control loop, optimized for the characteristics of HVAC system controls, including an additional exploration trick based on noise estimation and a mechanism to ensure constraint satisfaction. The performance of the proposed framework was evaluated by using a simulation system, which was calibrated by using trending data from a real data center. The results of our study showed that the proposed approach achieved more than a 10% increase in energy-efficiency savings within a few weeks of optimization time compared with the original building automation control.

Asynchronous Computation of Tube-based Model Predictive Control

Tube-based model predictive control (MPC) methods bound deviations from a nominal trajectory due to uncertainties in order to ensure constraint satisfaction. While techniques that compute the tubes online reduce conservativeness and increase performance, they suffer from high and potentially prohibitive computational complexity. This paper presents an asynchronous computation mechanism for system level tube-MPC (SLTMPC), a recently proposed tube-based MPC method which optimizes over both the nominal trajectory and the tubes. Computations are split into a primary and a secondary process, computing the nominal trajectory and the tubes, respectively. This enables running the primary process at a high frequency and moving the computationally complex tube computations to the secondary process. We show that the secondary process can continuously update the tubes, while retaining recursive feasibility of the primary process.

Distributionally robust model predictive control for wind farms

In this paper, we develop a distributionally robust model predictive control framework for the control of wind farms with the goal of power tracking and mechanical stress reduction of the individual wind turbines. We introduce an ARMA model to predict the turbulent wind speed, where we merely assume that the residuals are sub-Gaussian noise with statistics contained in an moment-based ambiguity set. We employ a recently developed distributionally model predictive control scheme to ensure constraint satisfaction and recursive feasibility of the control algorithm. The effectiveness of the approach is demonstrated on a practical example of five wind turbines in a row.

Safe-by-design control for Euler–Lagrange systems

Safety-critical control is characterized as ensuring constraint satisfaction for a given dynamical system. Recent developments in zeroing control barrier functions (ZCBFs) have provided a framework for ensuring safety of a superlevel set of a single constraint function. Euler–Lagrange systems represent many real-world systems including robots and vehicles, which must abide by safety-regulations, especially for use in human-occupied environments. These safety regulations include state constraints (position and velocity) and input constraints that must be respected at all times. ZCBFs are valuable for satisfying system constraints for general nonlinear systems, however their construction to satisfy state and input constraints is not straightforward. Furthermore, the existing barrier function methods do not address the multiple state constraints that are required for safety of Euler–Lagrange systems. In this paper, we propose a methodology to construct multiple, non-conflicting control barrier functions for Euler–Lagrange systems subject to input constraints to satisfy safety regulations, while concurrently taking into account robustness margins and sampling-time effects. The proposed approach consists of a sampled-data controller and an algorithm for barrier function construction to enforce safety (i.e. satisfy position and velocity constraints). The proposed method is validated in simulation on a 2-DOF planar manipulator.

Prescribed performance control of Euler–Lagrange systems with input saturation and output constraints

SummaryIn this article, we investigate the performance‐guaranteed tracking control for Euler–Lagrange (EL) systems subject to input saturation and output constraints. The prescribed performance and output constraints are first considered for EL systems with input saturation. Time‐varying barrier Lyapunov function (BLF) is employed to ensure constraint satisfaction. An adaptive control method for EL systems with saturation is put forward by adding a speed transformation into the BLF and designing auxiliary variables to the system. Subsequently, neural network is introduced to compensate for uncertainties of systems. In combination with backstepping technique, it is demonstrated that the controller not only guarantees that the tracking error converges to a preset compact set near zero at a specified decay rate, but also ensures that the output always stays within a given boundary. Simulation results further demonstrate the effectiveness of the proposed control scheme.

System level disturbance reachable sets and their application to tube-based MPC

Tube-based model predictive control (MPC) methods leverage tubes to bound deviations from a nominal trajectory due to uncertainties in order to ensure constraint satisfaction. This paper presents a novel tube‑based MPC formulation based on system level disturbance reachable sets (SL‑DRS), which leverage the affine system level parameterization (SLP). We show that imposing a finite impulse response (FIR) constraint on the affine SLP guarantees containment of all future deviations in a finite sequence of SL‑DRS. This allows us to formulate a system level tube‑MPC (SLTMPC) method using the SL‑DRS as tubes, which enables concurrent optimization of the nominal trajectory and the tubes, while using a positively invariant terminal set. Finally, we show that the SL‑DRS tubes can also be computed offline.

On Robustness in Optimization-Based Constrained Iterative Learning Control

Iterative learning control (ILC) is a control strategy for repetitive tasks wherein information from previous runs is leveraged to improve future performance. Optimization-based ILC (OB-ILC) is a powerful design framework for constrained ILC where measurements from the process are integrated into an optimization algorithm to provide robustness against noise and modelling error. This paper proposes a robust ILC controller for constrained linear processes based on the forward-backward splitting algorithm. It demonstrates how structured uncertainty information can be leveraged to ensure constraint satisfaction and provides a rigorous stability analysis in the iteration domain by combining concepts from monotone operator theory and robust control. Numerical simulations of a precision motion stage support the theoretical results.

Robust Predictive Energy Management of Connected Power-Split Hybrid Electric Vehicles Using Dynamic Traffic Data

Abstract This research focuses on the predictive energy management of connected human-driven hybrid electric vehicles (HEVs) to improve their fuel efficiency while robustly satisfying system constraints. We propose a hierarchical control framework that effectively exploits long-term and short-term decision-making benefits by integrating real-time traffic data into the energy management strategy. A pseudo-spectral optimal controller (PSOC) with discounted cost is utilized at the high level to find an approximate optimal solution for the entire driving cycle. At the low-level, a long short-term memory neural network (NN) is developed for higher quality velocity predictions over the low-level's short time horizons. Tube-based model predictive controller is then used at the low level to ensure constraints satisfaction in the presence of velocity prediction errors. Simulation results over real-world traffic data show an improvement in fuel economy for the proposed controller that is real-time applicable and robust to the driving cycle's uncertainty.

Constrained model-free reinforcement learning for process optimization

Reinforcement learning (RL) is a control approach that can handle nonlinear stochastic optimal control problems. However, despite the promise exhibited, RL has yet to see marked translation to industrial practice primarily due to its inability to satisfy state constraints. In this work we aim to address this challenge. We propose an “oracle”-assisted constrained Q-learning algorithm that guarantees the satisfaction of joint chance constraints with a high probability, which is crucial for safety critical tasks. To achieve this, constraint tightening (backoffs) are introduced and adjusted using Broyden’s method, hence making the backoffs self-tuned. This results in a methodology that can be imbued into RL algorithms to ensure constraint satisfaction. We analyze the performance of the proposed approach and compare against nonlinear model predictive control (NMPC). The favorable performance of this algorithm signifies a step towards the incorporation of RL into real world optimization and control of engineering systems, where constraints are essential.

Thrust vector control of constrained multibody systems

This paper presents a general control framework for the constrained control of multibody systems actuated by vectorized thrusters. A cascade control scheme augmented with a constraint-enforcement unit is proposed to stabilize the system while ensuring constraint satisfaction at all times. The cascade controller consists of an inner loop and an outer loop that are interconnected by a suitable mapping. The inner loop is tasked to control the attitude of the vectorized thrusters. The outer loop is designed to steer the task configuration of the system to a desired pose, while providing to the inner loop the desired attitude through a mapping. To prove the stability of the interconnected system, input-to-state stability (ISS) and small gain arguments are used. All stability properties are derived in the absence of constraints and are shown to be local. Hence, the control scheme is augmented with a Reference Governor (RG) to enforce constraints at all times. Simulations on an unmanned tiltrotor docked to a stationary platform for a refueling operation are carried out to demonstrate the effectiveness of the proposed control framework.

Constrained reference learning for continuous-time model predictive tracking control of autonomous systems

Abstract Often, systems need to adapt their behavior to other systems in their surroundings while obeying constraints to achieve good performance or due to safety reasons. We consider repetitive applications, where the reference for the controller stems from noisy sensor data. Including preview information of the reference, e.g. extrapolating from previous cycles or similar instances, can significantly improve the overall tracking performance and ensure constraint satisfaction. We propose a learning-supported predictive controller that uses Gaussian processes as reference generators for its control task. A Gaussian process is used to learn, filter, and predict the references. It learns references tailored to model predictive control, taking into account continuous-time system dynamics and constraints via constrained hyperparameter optimization. We illustrate the benefits concerning approximation and control performance of the informed Gaussian-process training by a cooperative, mobile robot example.

A nonlinear tracking model predictive control scheme for dynamic target signals

We present a nonlinear model predictive control (MPC) scheme for tracking of dynamic target signals. The scheme combines stabilization and dynamic trajectory planning in one layer, thus ensuring constraint satisfaction irrespective of changes in the dynamic target signal. For periodic target signals we ensure exponential stability of the optimal reachable periodic trajectory using suitable terminal ingredients and a convexity condition for the underlying periodic optimal control problem. Furthermore, we introduce an online optimization of the terminal set size to automate the trade-off between fast convergence and operation close to the constraints. In addition, we show how stabilization and dynamic trajectory planning can be formulated as partially decoupled optimization problems, which reduces the computational demand while ensuring recursive feasibility and convergence. The main tool to enable the proposed design is a novel reference generic offline computation that provides suitable terminal ingredients for tracking of dynamic reference trajectories. The practicality of this approach is demonstrated on benchmark examples, which demonstrates superior performance compared to state of the art approaches.

Tube-based Anticipative Robust MPC for Systems with Multiplicative Uncertainty

Many control problems contain time-varying or parameter-varying dynamics. With model predictive control (MPC), it is possible to include known plant variations into the controller for improving control performance. Unfortunately, perfect knowledge of the plant is rarely available and the accurateness of models may change over time and operating points. Robust control approaches consider worst-case realizations with a static model which ensure constraint satisfaction and stability but may yield conservative performance. The control algorithm presented in this paper uses anticipative information about future uncertainties and varying models to improve control performance while ensuring stability and feasibility. Possible system trajectories are bounded by polytopic tubes and recursive feasibility is achieved by the use of a terminal set. The controller properties are evaluated in a numerical example and compared to a similar control algorithm.