Bilevel Optimal Control Research Articles

A bi-level real-time optimal control strategy for thermal coupled multi-zone dedicated outside air system-assisted HVAC systems

This paper proposes a bi-level optimal control strategy for thermally coupled multi-zone dedicated outside air system (DOAS)-assisted HVAC systems, which integrates the deep reinforcement learning (DRL) algorithm with Lyapunov function-based reward shaping (LRS) and an augmented genetic algorithm. The proposed strategy aims to maintain indoor air quality (IAQ) and thermal comfort while reducing the overall energy cost. On the upper level, the LRS method enhances the convergence of the DRL algorithm, ensuring satisfactory IAQ with reduced electricity cost. On the lower level, the embedding of the pre-trained optimal agent achieves a unidirectional decoupling, which mitigates model dependency and accurately control thermal comfort under thermal coupling. The proposed strategy offers the following advantages: a) It can efficiently maintain IAQ in multiple zones and high energy efficiency by optimizing the supply airflow, in response to real-time environmental changes. b) It can achieve satisfactory thermal comfort by regulating the fan coil unit power, while implicitly reducing the overall energy costs via unidirectional decoupling. c) Experimental results demonstrate that the proposed strategy achieves a maximum energy savings of 33.13 % and electricity savings of 34.14 %, compared with the traditional strategy. d) The proposed strategy exhibits good generalization ability under different test climates and occupancy scenarios.

Optimality conditions for bilevel optimal control problems with non-convex quasi-variational inequalities

We establish Pontryagin optimality conditions for a generalized bilevel optimal control problem in which the leader is subject to a pure state inequality constraint, while the follower is governed by a non-convex quasi-variational inequality parameterized by the final state. To simplify the problem at hand, we convert it into a single-level optimal control problem by mapping the solution set of the quasi-variational inequality to a parametric optimization problem and employing the value function reformulation. Furthermore, we introduce certain regularity conditions to ensure that the derived maximum principle remains non-degenerate. Finally, we provide an illustrative example to elucidate our research findings.

Economic optimal control of source-storage collaboration based on wind power forecasting for transient frequency regulation

Under the dual crisis of climate and energy, clean energy has been rapidly developing, and the wind penetration of power systems has been continuously improving, which has brought much attention to the frequency safety of the power grid. This paper discusses two types of transient frequency regulation (TFR) scenarios with source-storage collaboration, where wind power and energy storage are used as auxiliary TFR resource. First, a distributed ultra-short-term wind power forecasting (WPF) method is proposed to facilitate the TFR resource planning of system operators. Second, a bi-level optimal control strategy is developed, aiming for the lowest TFR cost. According to the TFR power demand at the station level, the TFR powers of wind turbines, battery energy storage units, and conventional thermo-synchronous generators are dynamically allocated by using the accelerated alternating direction method of multipliers algorithm at the unit level. Finally, a case study is conducted to validate the effectiveness of the proposed WPF method and control strategy. Simulation results show that the WPF method ensures good prediction accuracy and qualification rate, and the control strategy effectively optimizes the economy of TFR.

Pontryagin optimality conditions for generalized bilevel optimal control problems with pure state inequality constraints

Pontryagin optimality conditions for generalized bilevel optimal control problems with pure state inequality constraints

Ecological Cooperative Adaptive Control of Connected Automate Vehicles in Mixed and Power-Heterogeneous Traffic Flow

The development of vehicle electrification and intelligent network technologies has led to a new type of mixed and power-heterogeneous traffic flow, comprised of regular vehicles (RVs) and connected and automated vehicles (CAVs), fuel vehicles (FVs) and electric vehicles (EVs). To reduce the energy consumption of mixed and power-heterogeneous traffic flow operating at a signalized intersection, the Ecological Control Unit–Cooperative Adaptive Control (ECU-CACC) is proposed in this paper. The vehicle platoon is divided into units which are named minimum ecological control units (min-ECUs). A bi-level control framework is designed to improve traffic efficiency and reduce energy consumption. The lower-level aims to plan the best ecological trajectory for every min-ECU, and the upper-level optimizes the passing strategies for efficiency through speed coordination. Scenario numerical experiments are performed to verify the effectiveness of the bi-level optimal control model and analyze the energy-saving effect of ECU-CACC under different vehicle mixing situations. The results from the experiment prove the excellent energy-saving potential of the proposed ECU-CACC, which helps the min-ECUs save about 10–20% energy consumption compared with a regular pattern.

Force Sharing Problem During Gait Using Inverse Optimal Control

Human gait patterns have been intensively studied, both from medical and engineering perspectives, to understand and compensate pathologies. However, the muscle-force sharing problem is still debated as acquiring individual muscle force measurements is challenging, requiring the use of invasive devices. Recent studies, using various objective functions, suggest muscle-force sharing may result from an optimization process. This study proposes using inverse optimal control to identify an objective function. Two popular methods of inverse optimal control, bilevel and inverse Karush-Kuhn-Tucker, were investigated. The identified objective functions were then used to predict muscle forces during gait, and their performances were compared to an exhaustive list of biological cost functions from the literature. The best prediction was achieved by the bilevel inverse optimal control method, with a root-mean-squared error of 176 N (162 N) and a correlation coefficient of 0.76 (0.68) for the stance (swing) phase of the gait cycle. These muscle force predictions were thereafter used to compute joint stiffness, exhibiting an average root-mean-square error of 42 Nm.rad <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{-1}$</tex-math></inline-formula> and a correlation coefficient of 0.90 when compared to the reference. The bilevel method's prevalence in terms of robustness over inverse Karush-Kuhn-Tucker was demonstrated on human data and explained on a toy example.

A Maximum Principle for a Time-Optimal Bilevel Sweeping Control Problem

In this article, we investigate a time-optimal state-constrained bilevel optimal control problem whose lower-level dynamics feature a sweeping control process involving a truncated normal cone. By bilevel, it is meant that the optimization of the upper level problem is carried out over the solution set of the lower level problem.This problem instance arises in structured crowd motion control problems in a confined space. We establish the corresponding necessary optimality conditions in the Gamkrelidze’s form. The analysis relies on the smooth approximation of the lower level sweeping control system, thereby dealing with the resulting lack of Lipschitzianity with respect to the state variable inherent to the sweeping process, and on the flattening of the bilevel structure via an exact penalization technique. Necessary conditions of optimality in the Gamkrelidze’s form are applied to the resulting standard approximating penalized state-constrained single-level problem, and the main result of this article is obtained by passing to the limit.

Bilevel optimal control of urban traffic-related air pollution by means of Stackelberg strategies

Air contamination and road congestion are two major problems in modern cities. Both are closely related and present the same source: traffic flow. To deal with these problems, governments impose traffic restrictions preventing the entry of vehicles into sensitive areas, with the final goal of decreasing pollution levels. Unfortunately, these restrictions force drivers to look for alternative routes that usually generate traffic congestions, resulting in longer travel times and higher levels of contamination. In this work, blending computational modelling and optimal control of partial differential equations, we formulate and analyse a bilevel optimal control problem with air pollution and drivers’ travel time as objectives and look for optimal solutions in the sense of Stackelberg. In this setting, the leader (local government) implements traffic restrictions meanwhile the follower (drivers set) acts choosing travel preferences against leader constraints. We discretize the problem and propose a numerical algorithm to solve it, combining genetic-elitist algorithms and interior-point methods. Finally, computational results for a realistic case posed in the Guadalajara Metropolitan Area (Mexico) are shown.

Real-Time Energy-Efficient Driver Advisory System for High-Speed Trains

An on-board energy-efficient driving advisory system is proposed for a high-speed train, which runs on a vehicle control unit. A data interaction framework is presented to monitor train real-time state and temporary speed limitations. A bilevel optimal control method is designed to calculate the energy-optimal speed profile in real time. Specifically, the upper level is to calculate the min-time speed profile based on Pontryagin’s maximum principle, and the lower level is to determine coast profiles according to the Lagrange multiplier technique. The driving advice is extracted from the optimized trajectories. If the deviation of the train trajectory is detected, the speed profile is updated online. To highlight the achievable improvements of the proposed optimal control method, a pseudospectral method and a quadratic programming method are introduced as performance comparison. Simulation results show that the proposed method yields an energy-optimal solution within a control cycle of 256 ms. The driver-in-the-loop tests prove that the driving advice can be exactly followed by drivers. The proposed driver advisory system can improve the performance of punctuality and energy efficiency.

COVID-9 and Unemployment: A Novel Bi-level Optimal Control Model

Since COVID-19 was declared as a pandemic in March 2020, the world’s major preoccupation has been to curb it while preserving the economy and reducing unemployment. This paper uses a novel Bi-Level Dynamic Optimal Control model (BLDOC) to coordinate control between COVID-19 and unemployment. The COVID-19 model is the upper level while the unemployment model is the lower level of the bi-level dynamic optimal control model. The BLDOC model’s main objectives are to minimize the number of individuals infected with COVID-19 and to minimize the unemployed individuals, and at the same time minimizing the cost of the containment strategies. We use the modified approximation Karush–Kuhn–Tucker (KKT) conditions with the Hamiltonian function to handle the bi-level dynamic optimal control model. We consider three control variables: The first control variable relates to government measures to curb the COVID-19 pandemic, i.e., quarantine, social distancing, and personal protection; and the other two control variables relate to government interventions to reduce the unemployment rate, i.e., employment, making individuals qualified, creating new jobs reviving the economy, reducing taxes. We investigate four different cases to verify the effect of control variables. Our results indicate that rather than focusing exclusively on only one problem, we need a balanced trade-off between controlling each.

Gradient-Based Solution Algorithms for a Class of Bilevel Optimization and Optimal Control Problems with a Nonsmooth Lower Level

The aim of this paper is to explore a peculiar regularization effect that occurs in the sensitivity analysis of certain elliptic variational inequalities of the second kind. The effect causes the solution operator of the variational inequality at hand to be continuously Fréchet differentiable, although the problem itself contains nondifferentiable terms. Our analysis shows in particular that standard gradient-based algorithms can be used to solve bilevel optimization and optimal control problems that are governed by elliptic variational inequalities of the considered type---all without regularizing the nondifferentiable terms in the lower-level problem and without losing desirable properties of the solution such as, e.g., sparsity. Our results can, for instance, be used in the optimal control of Casson fluids and in bilevel optimization approaches for parameter learning in total variation image denoising models.

Semivectorial bilevel programming versus scalar bilevel programming

ABSTRACTWe consider an optimistic semivectorial bilevel programming problem in Banach spaces. The associated lower level multicriteria optimization problem is assumed to be convex w.r.t. its decision variable. This property implies that all its weakly efficient points can be computed applying the weighted-sum-scalarization technique. Consequently, it is possible to replace the overall semivectorial bilevel programming problem by means of a standard bilevel programming problem whose upper level variables comprise the set of suitable scalarization parameters for the lower level problem. In this note, we consider the relationship between this surrogate bilevel programming problem and the original semivectorial bilevel programming problem. As it will be shown, this is a delicate issue as long as locally optimal solutions are investigated. The obtained theory is applied in order to derive existence results for semivectorial bilevel programming problems with not necessarily finite-dimensional lower level decision variables. Some regarding examples from bilevel optimal control are presented.

Free finite horizon LQR: A bilevel perspective and its application to model predictive control

We consider linear-quadratic optimal control problems with free final time and terminal state constraints and propose a solution procedure that is particularly useful for online feedback control in a model-predictive control (MPC) framework. The procedure avoids the standard time transformation, which transforms the problem into an equivalent but non-convex optimal control problem on a fixed time horizon. The transformed problem typically suffers from many local minima, which might cause instabilities in online optimization tasks like LQR or model-predictive control. To avoid this drawback of the time transformation we develop a method from the viewpoint of bilevel optimal control, which is beneficial especially in online control tasks. The novelty of the approach is the optimal final time tracking procedure, for which we exploit a property of the Hamiltonian function. To this end we show that within the continuous-time closed-loop controlled system the optimal final time linearly decreases in time, as one could intuitively expect. Finally, numerical experiments support the effectiveness of the proposed algorithm.

Necessary optimality conditions for a special class of bilevel programming problems with unique lower level solution

We consider a bilevel programming problem in Banach spaces whose lower level solution is unique for any choice of the upper level variable. A condition is presented which ensures that the lower level solution mapping is directionally differentiable, and a formula is constructed which can be used to compute this directional derivative. Afterwards, we apply these results in order to obtain first-order necessary optimality conditions for the bilevel programming problem. It is shown that these optimality conditions imply that a certain mathematical program with complementarity constraints in Banach spaces has the optimal solution zero. We state the weak and strong stationarity conditions of this problem as well as corresponding constraint qualifications in order to derive applicable necessary optimality conditions for the original bilevel programming problem. Finally, we use the theory to state new necessary optimality conditions for certain classes of semidefinite bilevel programming problems and present an example in terms of bilevel optimal control.

A Bilevel Approach for the Stochastic Optimal Operation of Interconnected Microgrids

Smart grid planning and control is becoming a theme of high interest in the last years. This is due to the presence of distributed generation, power from renewable resources and storage systems, to the different actors present over the territory, and to the difficulty of defining appropriate models for decision support. A bilevel optimal control scheme is proposed for grids characterized by renewable and traditional power production, bidirectional power flows, dynamic storage systems, and stochastic modeling issues. In this scheme, the upper level decision maker (UDM) views the lower level decision makers (LDMs) or microgrids as single nodes. In the statement of the UDM problem, the LDM control strategies are structurally and parametrically constrained inside a nonlinear optimization problem that includes load flow equations. Then, the LDMs can follow references from the UDM and use available information at the local level to solve a stochastic optimization problem. The proposed control architecture has been applied to a specific case study (Savona, Italy).

Bilevel Optimal Control, Equilibrium, and Combinatorial Problems with Applications to Engineering

Bilevel Optimal Control, Equilibrium, and Combinatorial Problems with Applications to Engineering

Bilevel Optimal Control With Final-State-Dependent Finite-Dimensional Lower Level

In this paper we discuss special bilevel optimal control problems where the upper level problem is an optimal control problem of ODEs with control and terminal constraints and the lower level problem is a finite-dimensional parametric optimization problem where the parameter is the final state of the state variable of the upper level. We tackle this problem using tools from nonsmooth analysis, optimization in Banach spaces, and bilevel programming to derive necessary optimality conditions of linearized Pontryagin-type.

Bilevel Optimal Control Problems with Pure State Constraints and Finite-dimensional Lower Level

This paper focuses on the development of optimality conditions for a bilevel optimal control problem with pure state constraints in the upper level and a finite-dimensional parametric optimization problem in the lower level. After transforming the problem into an equivalent single-level problem, we concentrate on the derivation of a necessary optimality condition of Pontryagin type. We point out some major difficulties arising from the bilevel structure of the original problem and its pure state constraints in the upper level leading to a degenerated maximum principle in the absence of constraint qualifications. Hence, we use a partial penalization approach and a well-known regularity condition for optimal control problems with pure state constraints to ensure the nondegeneracy of the derived maximum principle. Finally, we illustrate the applicability of the derived theory by means of a small example.

Necessary Optimality Conditions for Optimal Control Problems with Equilibrium Constraints

This paper introduces and studies the optimal control problem with equilibrium constraints (OCPEC). The OCPEC is an optimal control problem with a mixed state and control equilibrium constraint formulated as a complementarity constraint, and it can be seen as a dynamic mathematical program with equilibrium constraints. It provides a powerful modeling paradigm for many practical problems such as bilevel optimal control problems and dynamic principal-agent problems. In this paper, we propose weak, Clarke, Mordukhovich, and strong stationarities for the OCPEC. Moreover, we give some sufficient conditions to ensure that the local minimizers of the OCPEC are Fritz John--type weakly stationary, Mordukhovich stationary, and strongly stationary. Unlike Pontryagain's maximum principle for the classical optimal control problem with equality and inequality constraints, a counterexample shows that for general OCPECs there may exist two sets of multipliers for complementarity constraints. A condition under which these...

Weak and strong stationarity in generalized bilevel programming and bilevel optimal control

In this article, we consider a general bilevel programming problem in reflexive Banach spaces with a convex lower level problem. In order to derive necessary optimality conditions for the bilevel problem, it is transferred to a mathematical program with complementarity constraints (MPCC). We introduce a notion of weak stationarity and exploit the concept of strong stationarity for MPCCs in reflexive Banach spaces, recently developed by the second author, and we apply these concepts to the reformulated bilevel programming problem. Constraint qualifications are presented, which ensure that local optimal solutions satisfy the weak and strong stationarity conditions. Finally, we discuss a certain bilevel optimal control problem by means of the developed theory. Its weak and strong stationarity conditions of Pontryagin-type and some controllability assumptions ensuring strong stationarity of any local optimal solution are presented.