Certainty Equivalence Control Research Articles

Adaptive Certainty-Equivalence Control With Regulation-Triggered Finite-Time Least-Squares Identification

For general nonlinear control systems, we present a novel approach to adaptive control, which employs a certainty-equivalence (indirect) control law and an identifier with event-triggered updates of the plant parameter estimates, where the triggers are based on the size of the plant's state, and the updates are conducted using a nonrecursive least-squares estimation over certain finite time intervals, with updates employing delayed measurements of the state. With a suitable nonrestrictive parameter-observability assumption, our adaptive controller guarantees global stability, regulation of the plant state, and our identifier achieves parameter convergence, in finite time, even in the absence of persistent excitation, for all initial conditions other than those where the initial plant state is zero. The robustness of our event-triggered adaptive control scheme to vanishing and nonvanishing disturbances is verified in simulations with the assistance of a dead-zone-like modification of the update law. The major distinctions of our approach from supervisory adaptive schemes are that our approach is indirect and our triggering is related to the control objective (the regulation error). The major distinction from the classical indirect Lyapunov adaptive schemes based on tuning related to the regulation error is that our approach does not involve a complex redesign of the controller to compensate for the detrimental effects of rapid tuning on the transients by incorporating the update law into the control law. Instead, our approach allows for the first time to use a simple certainty-equivalence adaptive controller for general nonlinear systems.

Energy Saving and Interference Coordination in HetNets Using Dynamic Programming and CEC

Energy efficiency in cellular networks has gained great relevance due to the increasing power supply demands in new generation heterogeneous network (HetNet). On the other hand, interference coordination is a key aspect in HetNets resource management which directly affects the performance and energy consumption. Although these two aspect are intertwined, they have been studied separately so far. In this paper, we address the joint problem of energy saving and interference coordination in HetNets. We formulate the problem as a finite horizon Markov decision process (MDP) leveraging two facts: the user traffic demands usually follow periodic patterns, and the knowledge and prediction of network load is crucial in order to select efficient network configurations. Quality of Service (QoS) in the network is defined as the ratio of users meeting a requirement specified by the operator, and allows us to account for a minimum QoS requirement by including a constraint in the formulation of the MDP. To address this MDP, we propose an approximate dynamic programming (ADP) algorithm which selects energy efficient farsighted configurations with QoS guarantees achieving near optimal performance. This ADP algorithm is built upon: 1) the certainty equivalent control principle, which simplifies the complexity of the MDP, and 2) machine learning techniques: a neural network and a polynomial regressor which allow us to predict the QoS and the consumption of the network in advance. We evaluate our proposal in a LTE-A network simulator following the 3GPP guidelines, and the results obtained show that a joint control of the energy saving and interference coordination mechanisms results in a notably performance improvement compared to a disjoint control, in terms of both energy savings and QoS guarantees. Moreover, our proposal has the advantage of being adaptable to the operator QoS requirements.

Analysis of Deterministic Control and Its Improvements for an Inventory Problem with Multiproduct Batch Differentiation

We consider a multiperiod planning problem faced by a biopharmaceutical firm that must coordinate the production and allocation of batches of intermediate products to end products for multiple markets. This is a challenging problem to solve optimally, so we derive a theoretical bound on the performance of a deterministic control (DC) in which all random variables are replaced by their expected values and the corresponding deterministic optimization problem is solved. This is a variant of an approach that is widely used in practice. We show that while DC can perform poorly in some instances, simple reoptimizations of DC (i.e., certainty equivalent control (CEC)) improve the performance of DC. That said, the benefit of reoptimization decreases as the magnitude of demand variation increases. To address this, and also to provide guidance for heuristic design, we derive performance bounds for two additional heuristic controls—(1) open-loop feedback control (OLFC), which reoptimizes a sequence of stochastic programs with a complete demand distribution information, and (2) multipoint approximation control (MPAC), which solves a dynamic program using an approximate demand distribution that incorporates slightly more distributional information beyond simple expected values. We show that although OLFC outperforms CEC, its improvement can be limited. On the other hand, with carefully chosen approximate demand distributions, MPAC can significantly outperform OLFC. This suggests that, in our setting, fully capturing decision dynamics coupled with a small amount of additional distributional information beyond expected values is more useful than fully capturing the demand distribution but coupling that with insufficient modeling of decision dynamics. The online appendix is available at https://doi.org/10.1287/opre.2017.1647 .

Risk-averse Stochastic Nonlinear Model Predictive Control for Real-time Safety-critical Systems

Stochastic nonlinear model predictive control has been developed to systematically find an optimal decision with the aim of performance improvement in dynamical systems that involve uncertainties. However, most of the current methods are risk-neutral for safety-critical systems and depend on computationally expensive algorithms. This paper investigates on the risk-averse optimal stochastic nonlinear control subject to real-time safety-critical systems. In order to achieve a computationally tractable design and integrate knowledge about the uncertainties, bounded trajectories generated to quantify the uncertainties. The proposed controller considers these scenarios in a risk-sensitive manner. A certainty equivalent nonlinear model predictive control based on minimum principle is reformulated to optimise nominal cost and expected value of future recourse actions. The capability of proposed method in terms of states regulations, constraints fulfilment, and real-time implementation is demonstrated for a semi-autonomous ecological advanced driver assistance system specified for battery electric vehicles. This system plans for a safe and energy-efficient cruising velocity profile autonomously.

On Incomplete Learning and Certainty-Equivalence Control

We consider a dynamic learning problem where a decision maker sequentially selects a control and observes a response variable that depends on chosen control and an unknown sensitivity parameter. After every observation, the decision maker updates his or her estimate of the unknown parameter and uses a certainty-equivalence decision rule to determine subsequent controls based on this estimate. We show that under this certainty-equivalence learning policy the parameter estimates converge with positive probability to an uninformative fixed point that can differ from the true value of the unknown parameter; a phenomenon that will be referred to as incomplete learning. In stark contrast, it will be shown that this certainty-equivalence policy may avoid incomplete learning if the parameter value of interest “drifts away” from the uninformative fixed point at a critical rate. Finally, we prove that one can adaptively limit the learning memory to improve the accuracy of the certainty-equivalence policy in both static (estimation), as well as slowly varying (tracking) environments, without relying on forced exploration.

Modeling uncertain passenger arrivals in the elevator dispatching problem with destination control

An Elevator Group Control System (EGCS) assigns an elevator of a group to each passenger transportation request by solving a snapshot optimization problem, the Elevator Dispatching Problem (EDP). In the destination control, passengers register their destination floors in the elevator lobbies, after which the EGCS completes the assignment at once and is not allowed to change it later. Therefore, the EDP is formulated as a stochastic optimal control problem, where uncertain future passenger arrivals are modeled by a Poisson and a geometric Poisson process. The EDP is considered as a certainty equivalent controller in which the uncertain quantities are replaced by their expected values, and as a robust controller in which they take multiple values according to risk scenarios. Numerical experiments show that the expectations do not accurately predict EDP variables. The modeling with the geometric Poisson process results in better forecasting accuracy than with the Poisson process and many scenarios that closely match the realizations of the variables. Hence, the scenarios can be used as a basis for a robust EDP which simultaneously minimizes a passenger service quality criterion and its variation due to uncertain demand.

On Incomplete Learning and Certainty-Equivalence Control

We consider a dynamic learning problem where a decision maker sequentially selects a control and observes a response variable that depends on chosen control and an unknown sensitivity parameter. After every observation, the decision maker updates her/his estimate of the unknown parameter and uses a certainty-equivalence decision rule to determine subsequent controls based on this estimate. We show that under this certainty-equivalence learning policy the parameter estimates converge with positive probability to an uninformative fixed point that can differ from the true value of the unknown parameter; a phenomenon that will be referred to as incomplete learning. In stark contrast, it will be shown that this certainty-equivalence policy may avoid incomplete learning if the parameter value of interest “drifts away” from the uninformative fixed point at a critical rate. Finally, we prove that one can adaptively limit the learning memory to improve the accuracy of the certainty-equivalence policy in both static (estimation), as well as slowly varying (tracking) environments, without relying on forced exploration.

Optimal dual adaptive agile mobile wireless power control

Mobile wireless channels change persistently and rapidly, so power control needs to adapt similarly in order to save battery power and mitigate interference. Training signals in every data packet are used to achieve this in current systems, albeit in a suboptimal fashion from a power consumption perspective. This problem is used as the basis for posing and solving in detail an optimal dual adaptive control problem and then deriving heuristic controllers from this optimal solution. Dual refers to the joint requirements of the control signal to probe the system for parameter estimation and to regulate the total energy use; these are conflicting requirements which reveal the complexity of optimal stochastic control in general. The information state is defined, computed and explicitly incorporated into the optimization. Performance and computational load comparisons are made between: the optimal control, the certainty equivalence control, and a simplified heuristic approach. The contribution of the paper is the explicit solution of a classically hard optimal stochastic control problem to expose the role of the information state.

A theoretical stochastic control framework for adapting radiotherapy to hypoxia

Hypoxia, that is, insufficient oxygen partial pressure, is a known cause of reduced radiosensitivity in solid tumors, and especially in head-and-neck tumors. It is thus believed to adversely affect the outcome of fractionated radiotherapy. Oxygen partial pressure varies spatially and temporally over the treatment course and exhibits inter-patient and intra-tumor variation. Emerging advances in non-invasive functional imaging offer the future possibility of adapting radiotherapy plans to this uncertain spatiotemporal evolution of hypoxia over the treatment course.We study the potential benefits of such adaptive planning via a theoretical stochastic control framework using computer-simulated evolution of hypoxia on computer-generated test cases in head-and-neck cancer. The exact solution of the resulting control problem is computationally intractable. We develop an approximation algorithm, called certainty equivalent control, that calls for the solution of a sequence of convex programs over the treatment course; dose–volume constraints are handled using a simple constraint generation method. These convex programs are solved using an interior point algorithm with a logarithmic barrier via Newton’s method and backtracking line search. Convexity of various formulations in this paper is guaranteed by a sufficient condition on radiobiological tumor-response parameters. This condition is expected to hold for head-and-neck tumors and for other similarly responding tumors where the linear dose–response parameter is larger than the quadratic dose–response parameter.We perform numerical experiments on four test cases by using a first-order vector autoregressive process with exponential and rational-quadratic covariance functions from the spatiotemporal statistics literature to simulate the evolution of hypoxia. Our results suggest that dynamic planning could lead to a considerable improvement in the number of tumor cells remaining at the end of the treatment course. Through these simulations, we also gain insights into when and why dynamic planning is likely to yield the largest benefits.

Adaptive fuzzy PD control with stable H∞ tracking guarantee

For indirect adaptive fuzzy H∞ tracking control (AFHC) of perturbed uncertain nonlinear systems, sliding-mode control (SMC) compensation usually has to be applied to ensure stability and H∞ robustness of the closed-loop system. We prove that indirect AFHC without SMC compensation is sufficient to guarantee stable H∞ tracking under given initial conditions and parameter constraints. The control structure only includes an indirect adaptive fuzzy control term and a proportional derivative (PD) control term. A certainty equivalent control law is slightly modified such that both a lumped perturbation and adaptive laws are independent of the PD control term. This modification is significant since it not only plays a key role in stability analysis, but also alleviates some drawbacks of existing AFHC approaches for practical applications. An illustrative example has been provided to verify correctness of the theoretical result.

Structural Properties in a Hub-to-Hub Network Revenue Management Problem

In this paper, the structural properties of revenue management in a hub-to-hub airline network is studied. Using a reformulated network flow version of the problem, it is shown that the optimal value has supermodularity, submodularity, and \(L^\natural \) concavity in the network’s capacities dimensions. It is thus deduced that the certainty equivalent control thresholds used in the revenue management problem have monotone properties. These structural properties add important managerial insights into the network revenue management system.

Irrigation control based on model predictive control (MPC): Formulation of theory and validation using weather forecast data and AQUACROP model

This research proposes A THEORETICAL FRAMEWORK based on model predictive control (MPC) for irrigation control to minimize both root zone soil moisture deficit (RZSMD) and irrigation amount under a limited water supply. We (i) investigate means to incorporate direct measurements to MPC (ii) introduce two Robust MPC techniques – Certainty Equivalence control (CE) and Disturbance Affine Feedback Control (DA) – to mitigate the uncertainty of weather forecasts, and (iii) provide conditions to obtain two important theoretical aspects of MPC – feasibility and stability – in the context of irrigation control. Our results show that system identification enables automation while incorporating direct measurements. Both DA and CE minimize RZSMD and irrigation amount under uncertain weather forecasts and always maintain soil moisture above wilting point subject to water availability. The theoretical results are compared against the model AQUACROP, weather data and forecasts from Shepparton, Australia. We also discuss the performance of Robust MPC under different water availability, soil, crop conditions. In general, MPC shows to be a promising tool for irrigation control.

Certainty Equivalent Planning for Multi-Product Batch Differentiation: Analysis and Bounds

We consider a multi-period planning problem faced by a firm that must coordinate the production and allocations of batches to end products for multiple markets. Motivated by a problem faced by a biopharmaceutical firm, we model this as a discrete-time inventory planning problem where in each period the firm must decide how many batches to produce and how to differentiate batches to meet demands for different end products. This is a challenging problem to solve optimally, so we derive a theoretical bound on the performance of a Certainty Equivalent (CE) control for this model, in which all random variables are replaced by their expected values and the corresponding deterministic optimization problem is solved. This is a variant of an approach that is widely used in practice. We show that while a CE control can perform very poorly in certain instances, a simple re-optimization of the CE control in each period can substantially improve both the theoretical and computational performance of the heuristic, and we bound the performance of this re-optimization. To address the limitations of CE control and provide guidance for heuristic design, we also derive performance bounds for two additional heuristic controls -- (1) Re-optimized Stochastic Programming (RSP), which utilizes full demand distribution but limits the adaptive nature of decision dynamics, and (2) Multi-Point Approximation (MPA), which uses limited demand information to model uncertainty but fully capture the adaptive nature of decision dynamics. We show that although RSP in general outperforms the re-optimized CE control, the improvement is limited. On the other hand, with a carefully chosen demand approximation in each period, MPA can significantly outperform RSP. This suggests that, in our setting, explicitly capturing decision dynamics adds more value than simply capturing full demand information.

Dynamic Capacity Management with General Upgrading

This paper studies a capacity management problem with upgrading. A firm needs to procure multiple classes of capacities and then allocate the capacities to satisfy multiple classes of customers that arrive over time. A general upgrading rule is considered, i.e., unmet demand can be satisfied using multistep upgrade. No replenishment is allowed and the firm has to make the allocation decisions without observing future demand. We first characterize the structure of the optimal allocation policy, which consists of parallel allocation and then sequential rationing. Specifically, the firm first uses capacity to satisfy the same-class demand as much as possible, then considers possible upgrading decisions in a sequential manner. We also propose a heuristic based on certainty equivalence control to solve the problem. Numerical analysis shows that the heuristic is fast and delivers close-to-optimal profit for the firm. Finally, we conduct extensive numerical studies to derive insights into the problem. It is found that under the proposed heuristic, the value of using sophisticated multistep upgrading can be quite significant; however, using simple approximations for the initial capacity leads to negligible profit loss, which suggests that the firm’s profit is not sensitive to the initial capacity decision if the optimal upgrading policy is used.

Dynamic Capacity Management with General Upgrading

This paper studies a capacity management problem with upgrading. A firm needs to procure multiple classes of capacities and then allocate the capacities to satisfy multiple classes of customers that arrive over time. A general upgrading rule is considered, i.e., unmet demand can be satisfied using multi-step upgrade. No replenishment is allowed and the firm has to make the allocation decisions without observing future demand. We first characterize the structure of the optimal allocation policy, which consists of parallel allocation and then sequential rationing. Specifically, the firm first uses capacity to satisfy the same-class demand as much as possible, then considers possible upgrading decisions in a sequential manner. We also propose a heuristic based on certainty equivalence control to solve the problem. Numerical analysis shows that the heuristic is fast and delivers close-to-optimal profit for the firm. Finally, we conduct extensive numerical studies to derive insights into the problem. It is found that under the proposed heuristic, the value of using sophisticated multi-step upgrading can be quite significant; however, using simple approximations for the initial capacity leads to negligible profit loss, which suggests that the firm's profit is not sensitive to the initial capacity decision if the optimal upgrading policy is used.

Hybrid Certainty Equivalence Control of Rigid Bodies With Quaternion Measurements

In this technical note, we solve the control problem of rigid bodies with only quaternion measurements for all initial rotations and angular velocities. The proposed solution is based on the theory of cascades using any switching certainty equivalence controller satisfying certain assumptions along with an in the large hybrid observer. The equilibrium point of the proposed observer in closed loop with the rigid body dynamics is proven to be $\kappa$ -exponentially stable in the large i.e., we prove that the equilibrium point is stable and that the error states converge exponentially fast towards the origin for all initial rotations and angular velocities. Until now, stability results for quaternion-based observers have typically only been valid for a bounded set of initial conditions. To overcome this issue, our observer design is based on dynamic scaling and switching logic. Furthermore, we show that the origin of the proposed switching certainty equivalence controller in closed loop with the hybrid observer is asymptotically stable in the large for all available initial conditions associated with the quaternion space. Simulation results for the proposed scheme are presented with the particular case of the PD $+$ controller, revealing that all states converge as expected from our theoretical findings.

Optimal Coordination of G2V and V2G to Support Power Grids With High Penetration of Renewable Energy

Electric vehicles (EVs) have recently gained much popularity as a green alternative to fossil-fuel cars and a feasible solution to reduce air pollution in big cities. The use of EVs can also be extended as a demand response tool to support high penetration of renewable energy (RE) sources in future smart grid. Based on the certainty equivalent adaptive control (CECA) principle and a customer participation program, this paper presents a novel control strategy using optimization technique to coordinate not only the charging but also the discharging of EV batteries to deal with the intermittency in RE production. In addition, customer charging requirements and schedules are incorporated into the optimization algorithm to ensure customer satisfaction, and further improve the control performance. The merits of this scheme are its simplicity, efficiency, robustness and readiness for practical applications. The effectiveness of the proposed control algorithm is demonstrated by computer simulations of a power system with high level of wind energy integration.

Optimal minimum bids and inventory scrapping in sequential, single-unit, Vickrey auctions with demand learning

Retailers often conduct sequential, single-unit auctions and need to decide the minimum bid in each auction. To reduce inventory costs, it may be optimal to scrap some of the inventory rather than holding it until it is auctioned off. In some auctions, the seller may be uncertain about the market response and hence may want to dynamically learn the demand by observing the number of posted bids. We formulate a Markov decision process (MDP) to study this dynamic auction-design problem under the Vickrey mechanism.We first develop a clairvoyant model where the seller knows the demand distribution. We prove that it is optimal to scrap all inventory above a certain threshold and then auction the remaining units. We derive a first order necessary condition whereby the bidders’ virtual value at an optimal minimum bid equals the seller’s marginal profit. This is a generalization of Riley and Samuelson’s result from the one, single-unit auction case. When the virtual value is strictly increasing, this necessary condition is also sufficient and leads to a structured value iteration algorithm.We then assume that the number of bidders is Poisson distributed but the seller does not know its mean. The seller uses a mixture-of-Gamma prior on this mean and updates this belief over several auctions. This results in a high-dimensional Bayesian MDP whose exact solution is intractable. We therefore propose and compare two approximation methods called certainty equivalent control (CEC) and Q-function approximation. Numerical experiments suggest that Q-function approximation can attain higher revenues than CEC.

Wing rock control by finite-form adaptation

The wing rock (self-excited rolling motion) of aircraft flying at high angle of attack adversely affects the handling qualities and maneuverability. This paper presents new adaptation algorithms in finite (integral-algebraic) form for the wing rock control. The objective is to stabilize the rolling motion, despite parameter uncertainties. Two adaptive control laws in finite form are derived for the trajectory control of the roll angle. The first control system is obtained by the realization of a speed-gradient adaptation law, and an immersion and invariance-based adaptive law is used for the second controller. Unlike the certainty-equivalent control laws, these adaptation laws include an integral term as well as a judiciously chosen nonlinear algebraic vector function. The algebraic function in the adaptation law provides stronger stability properties in the closed-loop system. Simulation results show that the designed adaptive controllers are capable of suppressing the wing rock motion of the model, despite uncertainties in the parameters.

Dynamic Auctions with Demand Learning

The minimum bid is a key design variable in retail auctions as it affects the number of posted bids and the clearing price. Retailers often conduct sequential, single-unit auctions online as a revenue generation and inventory clearing tool, and need to dynamically decide the minimum bid in each auction depending on the inventory level at that time. To reduce inventory costs, it may in fact be optimal to scrap some of the inventory rather than holding it until it is auctioned off. In auctions of fashion goods, seasonal products, new products, and where the seller is new to the market, the seller may be uncertain about the market response. Such a seller may want to dynamically learn the demand by observing the number of posted bids. This introduces an additional, exploration versus exploitation trade-off. We formulate a Markov decision process (MDP) to study this dynamic auction-design problem. We consider a Vickrey mechanism, and a random number of bidders with private, independent, and identically distributed valuations. We first develop a clairvoyant model where the seller knows the demand distribution. We prove that it is optimal to scrap all inventory above a certain threshold and then auction the remaining units. We derive a first order necessary condition, which states that, in any inventory level, the bidders' virtual value (as defined by Myerson) at an optimal minimum bid should equal the seller's marginal profit. This is an intuitive generalization of Riley and Samuelson's well-known result from the one single-unit auction case. Under the standard regularity condition that the virtual value is strictly increasing, our first order necessary condition is also sufficient and leads to a structured value iteration algorithm for efficiently computing optimal policies. In that case, we prove that optimal minimum bids (i) decrease with increasing inventory levels, and (ii) are higher than optimal minimum bids in one single-unit auction. We also prove that optimal scrapping thresholds and optimal minimum bids both decrease with increasing inventory costs.We illustrate our results and perform sensitivity analyses using examples with a Poisson-distributed number of bidders and Beta-distributed private valuations.In the second half of the paper, we assume that the number of bidders is Poisson distributed but the seller does not know its mean. The seller uses a mixture-of-Gamma prior on this mean and updates this belief over the sequence of auctions while simultaneously deciding minimum bids and scrapping quantities. This results in a high-dimensional Bayesian MDP whose exact solution is intractable. We therefore implement a certainty equivalent control (CEC) heuristic instead. We extend our clairvoyant structural result to CEC and numerically investigate the effect of initial inventory and mean demand on CEC performance.