Stochastic differential dynamic programming for multi-reservoir system control

Abstract

: As with all dynamic programming formulations, differential dynamic programming (DDP) successfully exploits the sequential decision structure of multi-reservoir optimization problems, overcomes difficulties with the nonconvexity of energy production functions for hydropower systems, and provides optimal feedback release policies. DDP is particularly well suited to optimizing large-scale multi-reservoir systems due to its relative insensitivity to state-space dimensionality. This advantage of DDP encourages expansion of the state vector to include additional multi-lag hydrologic information and/or future inflow forecasts in developing optimal reservoir release policies. Unfortunately, attempts at extending DDP to the stochastic case have not been entirely successful. A modified stochastic DDP algorithm is presented which overcomes difficulties in previous formulations. Application of the algorithm to a four-reservoir hydropower system demonstrates its capabilities as an efficient approach to solving stochastic multi-reservoir optimization problems. The algorithm is also applied to a single reservoir problem with inclusion of multi-lag hydrologic information in the state vector. Results provide evidence of significant benefits in direct inclusion of expanded hydrologic state information in optimal feedback release policies.