Constrained Differential Dynamic Programming Research Articles

Data-driven model predictive control design for offset-free tracking of nonlinear systems

We propose a design of data-driven Model Predictive Control (MPC) using a suboptimal trajectory and the linear time-varying (LTV) models from data-driven trajectory optimisation that achieves offset-free tracking. Data-driven constrained differential dynamic programming (CDDP) is exploited to improve the trajectory iteratively without the knowledge of the nonlinear model. A trajectory is divided to the transient and steady state regions, controlled by the Linear time-varying MPC (LTVMPC) and the offset-free linear MPC (LMPC), respectively. We prove the feasibility of the proposed LTVMPC in the transient region, and the offset-free tracking property of LMPC. The proposed scheme is validated to a continuous stirred tank reactor (CSTR) process. Simulation studies show that the suboptimal trajectory and LTV models are generated by CDDP, and the proposed MPC achieves offset-free tracking and disturbance rejection for a set of initial conditions and set points in the operating region.

Data-driven offset-free multilinear model predictive control using constrained differential dynamic programming

Multilinear model predictive control (MLMPC) can regulate a nonlinear process with wide operating regions based on a set of linear models. Although online computational cost is reduced compare to nonlinear MPC (NMPC), it is difficult to obtain a reliable full nonlinear model or set of linear models in practice. In this paper, we propose a combination of MLMPC with differential dynamic programming (DDP), so that the system can be controlled offset-free in the absence of a full nonlinear model. DDP is a ‘trajectory-centric’ optimization technique that solves nonlinear optimal control problems. The trajectory can be optimized even if the full model for the system is unknown, because DDP uses only the gradients around the visited trajectory, which is easily obtained by input excitations. Moreover, the gradient information can provide linear models in the subsequent MLMPC step. In the proposed scheme, a novel model selection based on gap metric and weighting method are employed for MLMPC. We prove the offset tracking property of DDP assisted MLMPC. A continuous stirred tank reactor (CSTR) process is studied to demonstrate the effectiveness of the proposed algorithms. Simulation studies show that CDDP designed by the proposed algorithm improves the trajectory over iterations, and the resulting MLMPC achieves offset-free tracking property regardless of an initial point and a set-point in the operating region.

Integration of Optimal Dynamic Control and Neural Network for Groundwater Quality Management

This study integrates an artificial neural network (ANN) and constrained differential dynamic programming (CDDP) to search for optimal solutions to a nonlinear time-varying groundwater remediation-planning problem. The proposed model (ANN-CDDP) determines optimal dynamic pumping schemes to minimize operating costs and meet water quality requirements. The model uses two embedded ANNs, including groundwater flow and contaminant transport models, as transition functions to predict groundwater levels and contaminant concentrations under time-varying pumping. Results demonstrate that ANN-CDDP is a simplified management model that requires considerably less computation time to solve a fine mesh problem. For example, the ANN-CDDP computing time for a case involving 364 nodes is 1/26.5 that of the conventional optimization model.

Optimizing Capacity-Expansion Planning of Groundwater Supply System between Cost and Subsidence

This work solves an optimal capacity-expansion planning problem with land subsidence constraints using a hybrid algorithm that combines the genetic algorithm (GA) and constrained differential dynamic programming (CDDP). The main structure of the hybrid algorithm is the GA, in which each chromosome represents a possible network design and its expansion schedule. The present fixed cost of each chromosome is computed easily using the GA, and CDDP is then used to solve the optimal pumping rates and compute the optimal present operating costs associated with the chromosome. Simulation results indicate that the well network has lower total present cost with the capacity-expansion system instead of installing the full system capacity at the beginning. However, the well network designed by the capacity-expansion model without considering land subsidence may induce more local land subsidence than that determined by the conventional model demonstrating the necessity of considering land subsidence constraints in the system design. This work also investigates other important issues related to the optimal design of the system-installing schedule. The proposed model is highly promising for facilitating a cost-efficient well system design for regional groundwater supply and environmental conservation.

Optimal control algorithm and neural network for dynamic groundwater management

AbstractResearchers have found that obtaining optimal solutions for groundwater resource‐planning problems, while simultaneously considering time‐varying pumping rates, is a challenging task. This study integrates an artificial neural network (ANN) and constrained differential dynamic programming (CDDP) as simulation‐optimization model, called ANN‐CDDP. Optimal solutions for a groundwater resource‐planning problem are determined while simultaneously considering time‐varying pumping rates. A trained ANN is used as the transition function to predict ground water table under variable pumping conditions. The results show that the ANN‐CDDP reduces computational time by as much as 94·5% when compared to the time required by the conventional model. The proposed optimization model saves a considerable amount of computational time for solving large‐scale problems. Copyright © 2009 John Wiley & Sons, Ltd.

Optimizing system capacity expansion schedules for groundwater supply

Groundwater plays a vital role in regional water resources management. In conventional design, a full‐scale network system is installed initially to use groundwater resources. However, the system capacity may exceed water demand in the early stages because water demand generally increases with time. Therefore, this work proposes a novel optimal capacity expansion model capable of determining an optimal schedule to expand system capacity according to increasing water demand. The proposed algorithm hybridizes a genetic algorithm (GA) and constrained differential dynamic programming (CDDP). The chromosomes of the GA represent a possible design alternative, a groundwater network with capacity that expands with time. The CDDP algorithm is then used to compute the optimal pumping policy associated with the chromosome. Simulation results indicate that the capacity expansion model saves more total present value cost than conventional designs for the same annual interest rate and water demand. Results of this study demonstrate promise for the proposed model in facilitating a cost‐effective groundwater network design with capacity expansion for regional groundwater supply.

Multi-objective Planning for Conjunctive Use of Surface and Subsurface Water Using Genetic Algorithm and Dynamics Programming

The consideration of fixed cost and time-varying operating cost associated with the simultaneous conjunctive use of surface and subsurface water should be treated as a multi-objective problem due to the conflicting characteristics of these two objectives. In order to solve this multi-objective problem, a novel approach is developed herein by integrating the multi-objective genetic algorithm (MOGA), constrained differential dynamic programming (CDDP) and the groundwater simulation model ISOQUAD. A MOGA is used to generate the various fixed costs of reservoirs’ scale, generate a pattern of pumping/recharge, and estimate the non-inferior solutions set. A groundwater simulation model ISOQUAD is directly embedded to handle the complex dynamic relationship between the groundwater level and the generated pumping/recharge pattern. The CDDP optimization model is then adopted to distribute the optimal releases among reservoirs provided that reservoir capacities are known. Finally, the effectiveness of our proposed integrated model is verified by solving a water resources planning problem for the conjunctive use of surface and subsurface water in southern Taiwan.

Application of Optimal Control and Fuzzy Theory for Dynamic Groundwater Remediation Design

Obtaining optimal solutions for time-varying groundwater remediation design is a challenging task. A novel procedure first employs input/output data sets obtained by constrained differential dynamic programming (CDDP). Then the Adaptive-Network-Based Fuzzy Inference System (ANFIS), which is a fuzzy inference system (FIS) implemented in the adaptive network framework, is applied to acquire time-varying pumping rates. Results demonstrate that the FIS is an efficient way of groundwater remediation design.

Optimal planning of a dynamic pump-treat-inject groundwater remediation system

This study integrates the genetic algorithm (GA) and constrained differential dynamic programming (CDDP) to design the pump-treat-inject system. The proposed model considers both the cost of installing wells (fixed cost) and the operating cost of pumping, injection and water treatment. To minimize the total cost while meeting the water quality constraints, the model can compute the optimal number and locations of wells, as well as the associated optimal pumping and injection schemes. Various numerical cases reveal that the requirement to balance the total volume between pumping and injection can significantly influence the final optimal design.

Optimizing Remediation of an Unconfined Aquifer Using a Hybrid Algorithm

We present a novel hybrid algorithm, integrating a genetic algorithm (GA) and constrained differential dynamic programming (CDDP), to achieve remediation planning for an unconfined aquifer. The objective function includes both fixed and dynamic operation costs. GA determines the primary structure of the proposed algorithm, and a chromosome therein implemented by a series of binary digits represents a potential network design. The time-varying optimal operation cost associated with the network design is computed by the CDDP, in which is embedded a numerical transport model. Several computational approaches, including a chromosome bookkeeping procedure, are implemented to alleviate computational loading. Additionally, case studies that involve fixed and time-varying operating costs for confined and unconfined aquifers, respectively, are discussed to elucidate the effectiveness of the proposed algorithm. Simulation results indicate that the fixed costs markedly affect the optimal design, including the number and locations of the wells. Furthermore, the solution obtained using the confined approximation for an unconfined aquifer may be infeasible, as determined by an unconfined simulation.

Dynamic optimal ground water remediation including fixed and operation costs.

In time-varying ground water remediation, the lack of an optimal control algorithm to simultaneously consider fixed costs and time-varying operating costs makes it nearly impossible to obtain an optimal solution. This study presents a novel algorithm that integrates a genetic algorithm (GA) and constrained differential dynamic programming (CDDP) to solve this time-varying ground water remediation problem. A GA can easily incorporate the fixed costs associated with the installation of wells. However, using a GA to solve for time-varying policies would dramatically increase the computational resources required. Therefore, the CDDP is used to handle the subproblems associated with time-varying operating costs. A hypothetical case study that incorporates fixed and time-varying operating costs is presented to demonstrate the effectiveness of the proposed algorithm. Simulation results indicate that the fixed costs can significantly influence the number and locations of wells, and a notable total cost savings can be realized by applying the novel algorithm herein.

Nonlinear weighted feedback control of groundwater remediation under uncertainty

Differential dynamic programming is used to compute optimal time‐varying pumping policies for a pump and treat strategy for groundwater remediation. The feedback law generated by a constrained differential dynamic programming algorithm with penalty functions is used as the basis of feedback laws tested in cases where there is uncertainty in the hydraulic conductivity. Confined transient aquifer flow and transport are modeled using a two‐dimensional Galerkin finite element scheme with implicit time differencing. Optimal policies are calculated using a given or “measured” set of hydraulic conductivities and initial conditions. The optimal policies (with and without feedback) are applied using the same finite element model with a second or “true” set of conductivities. The “true” sets of conductivities are generated randomly from an autocorrelated lognormal distribution by the spectral method. The approach used here has an advantage over other uncertainty approaches because it is not necessary to specify precisely which parameters are considered uncertain and which are certain. Also no single probability distribution need be assumed for each uncertain parameter. By adjusting the relative weight assigned each penalty function, robust feedback laws were obtained that perform equally well under nine different assumed error distributions. In our examples, well‐designed feedback policies cost between 4% and 51% less than the cost of applying the calculated optimal policies without using a feedback law.

Constrained differential dynamic programming and its application to multireservoir control

This paper describes a modification of differential dynamic programming (DDP) which makes that technique applicable to certain constrained sequential decision problems such as multireservoir control problems discussed in the hydrology literature. The authors contend that the method proffered here is superior to available alternatives. This belief is supported by analysis (wherein it transpires that constrained DDP does not suffer the ‘curse of dimensionality’ and requires no discretization) and computational experimentation (wherein DDP is found to quickly locate solutions of 4‐reservoir problems introduced by other investigations as well as the solution of a 10‐reservoir problem thought to be beyond the capability of alternative methods).