Reservoir Operation Decisions Research Articles

Fill-and-Spill: Deep Reinforcement Learning Policy Gradient Methods for Reservoir Operation Decision and Control

Fill-and-Spill: Deep Reinforcement Learning Policy Gradient Methods for Reservoir Operation Decision and Control

Evolutionary algorithm-based multiobjective reservoir operation policy optimisation under uncertainty

Reservoir operation optimisation is a decision support tool to assist reservoir operators with water release decisions to achieve management objectives, such as maximising water supply security, mitigating flood risk, and maximising hydroelectric power generation. The effectiveness of reservoir operation decisions is subject to uncertainty in system inputs, such as inflow and therefore, methods such as stochastic dynamic programming (SDP) have been traditionally used. However, these methods suffer from the three curses of dimensionality, modelling, and multiple objectives. Evolutionary algorithm (EA)-based simulation-optimisation frameworks such as the Evolutionary Multi-Objective Direct Policy Search (EMODPS) offer a new paradigm for multiobjective reservoir optimisation under uncertainty, directly addressing the shortcomings of SDP-based methods. They also enable the consideration of input uncertainty represented using ensemble forecasts that have become more accessible recently. However, there is no universally agreed approach to incorporate uncertainty into EA-based multiobjective reservoir operation policy optimisation and it is not clear which approach is more effective. Therefore, this study conducts a comparative analysis to demonstrate the advantages and limitations of different approaches to account for uncertainty in multiobjective reservoir operation policy optimisation via a real-world case study; and provide guidance on the selection of appropriate approaches. Based on the results obtained, it is evident that each approach has both advantages and limitations. A suitable approach needs to be carefully selected based on the needs of the study, e.g., whether a hard constraint is required, or a well-established decision-making process exists. In addition, potential gaps for future research are identified.

Network flow and flood routing model for water resources optimization

Real-time management of hydraulic systems composed of multi-reservoir involves conflicting objectives. Its representation requires complex variables to consider all the systems dynamics. Interfacing simulation model with optimization algorithm permits to integrate flow routing into reservoir operation decisions and consists in solving separately hydraulic and operational constraints, but it requires that the water resource management model is based on an evolutionary algorithm. Considering channel routing in optimization algorithm can be done using conceptual models such as the Muskingum model. However, the structure of algorithms based on a network flow approach, inhibits the integration of the Muskingum model in the approach formulation. In this work, a flood routing model, corresponding to a singular form of the Muskingum model, constructed as a network flow is proposed and integrated into the water management optimization. A genetic algorithm is involved for the calibration of the model. The proposed flood routing model was applied on the standard Wilson test and on a 40 km reach of the Arrats river (southwest of France). The results were compared with the results of the Muskingum model. Finally, operational results for a water resource management system including this model are illustrated on a rainfall event.

Long-Term Scheduling of Cascade Reservoirs Considering Inflow Forecasting Uncertainty Based on a Disaggregation Model

Forecasted inflow is one of the most important input information for reservoir operation planning. However, current inflow prediction accuracy is difficult to meet the needs of long-term operation. Therefore, it is of great significance to consider the uncertainty of inflow forecast and study its influence on reservoir operation decisions. In this paper, a long-term scheduling framework considering inflow forecast uncertainty based on a temporal disaggregation method is proposed. First, the uncertainty of forecast is described from two aspects. Gaussian distribution is used to simulate the annual forecast error, and an Adaptive Nearest Neighbor Gaussian sampling method (A-NGS) is proposed to decompose annual inflow into temporal series. Based on the implicit scheduling model, the sample set of generation scheduling plans, actual dispatching schemes and theoretical optimal results are constructed. On this basis, a series of indexes are presented to evaluate the inflow simulation performance and the scheduling benefits. A case study of the Xiluodu-Xiangjiaba cascade reservoirs is conducted to analyze the effects of forecast uncertainty on operation benefits, and the effectiveness of forecast information is identified. Compared with the deterministic fragment method, the inflow processes simulated by A-NGS achieve better precision and behave more conducive to the scheduling. Although the uncertainty of forecast errors will bring some hydropower generation losses, a certain degree of forecast accuracy is effective to improve scheduling benefits when in the electricity market.

Feedback Between Reservoir Operation and Floodplain Development: Implications for Reservoir Benefits and Beneficiaries

AbstractCurrently, reservoir operation researchers and decision makers weigh the trade‐off between water conservation and flood control at the event and seasonal scales but have neglected the role of socioeconomic dynamics which unfold over longer time scales. However, ongoing advances in understanding human‐flood interactions provide an exemplary opportunity to diverge from this traditional treatment and instead incorporate feedbacks from downstream floodplain development in reservoir operation decisions. To do so, we modify and couple existing mathematical frameworks for optimal reservoir operation and leveed floodplain dynamics. The new coupled model demonstrates that, where reservoir operation persistently maximizes event‐ to seasonal‐scale benefits (i.e., the status quo), most reservoir‐floodplain systems will coevolve toward states of high preference for flood protection and highly developed floodplains. Further, under the status quo operation strategy, less benefits can be achieved as time goes on, benefits shift from water conservation consumers to floodplain inhabitants, and risk is transferred from floodplain inhabitants to water conservation consumers. We compare the status quo operation strategy to a strategy which stabilizes the floodplain development level and show that allowing excess (i.e., more than is optimal at the seasonal scale) flooding now preserves future water conservation benefits by disincentivizing floodplain development. Finally, we illustrate this coevolution and its ramifications at two U.S. reservoir‐floodplain systems where floodplain development has seemingly undermined hydropower production. The insights from this study fill a knowledge gap for floodplain managers and reservoir operators tasked with achieving economically efficient use of floodplains and reservoir storage.

Adapting reservoir operations to the nexus across water supply, power generation, and environment systems: An explanatory tool for policy makers

Reservoir operation plays an important role in the coevolution of the nexus across water supply, power generation, and environment (WPE) systems. This paper aims to contribute to the state-of-the-art in how multi-objective reservoir operation can be modeled within the framework of the WPE nexus by incorporating the interactions among reservoirs operation, stakeholders’ decision preferences, and WPE systems. Specifically, five ordinary differential equations are used to describe the dynamics of WPE states. These dynamics are driven by a multi-objective optimization model that optimizes reservoir operation decisions. Two novel response linkages that describe how the reservoir operational decisions influence socio-economic variables and how the environmental awareness influences stakeholders’ preference for reservoir operation are proposed in this study. The Middle Route of China’s South-to-North Water Transfer Project is selected as the case study. The results show that: (1) response linkages are effective in describing the interactions between reservoir operational decisions and WPE systems, (2) tradeoffs across WPE systems and reservoir operational decisions shift from stage to stage because of the changing socio-economic conditions, and (3) the control parameter that dominates the shift in stable levels of a given variable can be identified. These results highlight the importance of re-evaluating involved tradeoff effects and updating of operational policies periodically, and how our modeling approach can be used to detect and guide such adaptions.

Reservoir Operation by Artificial Neural Network Model ( Mosul Dam –Iraq, as a Case Study)

Reservoir operation forecasting plays an important role in managing water resources systems. Artificial Neural Network (ANN) model was applied for Mosul-Dam reservoir which is located on Tigris River, which the objectives of water resources development and flood control. Feed-forward multi-layer perceptions (MLPs) are used and trained with the back-propagation algorithm, as they have a high capability of data mapping. The data set has a period of 23 years from 1990 to 2012..The Input data were inflow (It), evaporation (Et), rainfall (Rt), reservoir storage (St) and outflow (Ot). The best convergence after more than 1000 trials was achieved for the combination of inflow (It), inflow (It-1), inflow (It-2), evaporation (Et), reservoir storage (St), rainfall (Rt), outflow (Ot-1) and outflow (Ot-2) with error tolerance, learning rate, momentum rate, number of cycles and number of hidden layers as 0.001, 1, 0.9,50000 and 9 respectively. The coefficient of determination (R2) and MAPE were (0.972) and (17.15) respectively. The results of ANN models for the training, testing and validation were compared with the observed data. The predicted values from the neural networks matched the measured values very well. The application of ANN technique and the predicted equation by using the connection weights and the threshold levels, assist the reservoir operation decision and future updating, also it is an important Model for finding the missing data. The ANN technique can accurately predict the monthly Outflow.

论水库运用设计洪水

计算水库运用设计洪水必须首先分析水库运用面临径流年内随机过程概率规律，并考虑水库调度决策特点、要求。本文为水库运用设计洪水预报概率模型提供了理论框架。并以三峡水库为例予以验证。结果表明，概率模型给出的三峡水库动态设计洪水预报是合理的，可靠的。 To calculate the design flood for reservoir operation, we first need to analyze the regularity of annual runoff process’s stochastic probability of the reservoir, and take the characteristics and the requirements of reservoir operation decision into account. A theoretical framework for develop-ment of design flood forecast probability model has been provided in this paper and validated with the Three Gorges Reservoir. The results indicate that the dynamic design flood forecast of the Three Gorges Reservoir based on this probability model is reasonable and reliable.

Multi-reservoir real-time operation rules: a new genetic programming approach

This paper employs non-linear programming, genetic algorithms and fixed-length gene genetic programming (FLGGP) for the real-time operation of a three-reservoir system (Karoon4, Khersan1 and Karoon3) in which dependent and independent approaches are used to forecast the hydroelectric energy generated by the system. A total deficiency function as well as efficiency criteria are used to investigate the results obtained. The latter indicate that the more flexible FLGGP gives the most efficient function for the extraction of reservoir operation rules in both dependent and independent approaches. By comparing the two approaches, no significant difference was observed. Consequently, due to the simplicity of the application of the forecast-independent approach, it is suggested for application in the extraction of reservoir operation decision rules. Moreover, the advantages of a three-reservoir system operation over a single-reservoir system operation reflect the efficiency of the integrated management of water resource systems.

Optimizing Reservoir Operation Policy Using Chance Constraint Nonlinear Programming for Koga Irrigation Dam, Ethiopia

One of typical problems in water resources system modeling is derivation of optimal operating policy for reservoir to ensure water is used more efficiently. This paper introduces optimization analysis to determine monthly reservoir operating policies for five scenarios of predetermined cropping patterns for Koga irrigation scheme, Ethiopia. The objective function of the model was set to minimize the sum of squared deviation (SSD) from the desired targeted supply. Reservoir operation under different water availability and thresholds of irrigation demands has been analyzed by running a chance constraint nonlinear programming model based on uncertain inflow data. The model was optimized using Microsoft Excel Solver. The lowest SSD and vulnerability, and the highest volumetric reliability were gained at irrigation deficit thresholds of 20 % under scenario I, 30 % under scenario II, III and V, and at 40 % under scenario IV when compensation release is permitted for downstream environment. These thresholds of deficits could be reduced by 10 % for all scenarios if compensation release is not permitted. In conclusion the reservoir water is not sufficient enough to meet 100 % irrigation demand for design command areas of 7,000 ha. The developed model could be used for real time reservoir operation decision making for similar reservoir irrigation systems. In this specific case study system, attempt should be made to evaluate the technical performance of the scheme and introduce a regulated deficit irrigation application.

Operating Rules Derivation of Jinsha Reservoirs System with Parameter Calibrated Support Vector Regression

The reservoir optimal operation depends on not only specific characteristics of reservoirs and hydropower stations but also stochastic inflows. The key issue of actual hydropower operation is to make an approximate optimal decision triggered by limited inflow forecasts. To implement actual optimal operation of hydropower system with limited inflows forecast, this paper makes use of Support Vector Regression (SVR) to derive optimal operating rules. To improve the performance of SVR, parameters in SVR model are calibrated with grid search and cross validation techniques. The trained SVR model describes the complex nonlinear relationships between reservoir operation decisions and factors by considering both generalization and regression performance, which overcomes local optimization and over fitting deficits. Hybrid programming platform is further developed to implement system simulation. This SVR model along with simulation platform is applied to the largest hydropower base in China – Jinsha system. Three scenarios are developed for comparison: deterministic optimal operation, SVR based simulation with calibrated parameters, SVR based simulation with default parameters. Comprehensive evaluation indicates that, operating rules derived from SVR presents a reliable performance in system power generation and output processes with respect to ideal deterministic results, especially when the parameters are calibrated. Hybrid programming technique provides a feasible and compatible platform for future research.

Identifying effective forecast horizon for real‐time reservoir operation under a limited inflow forecast

The use of a streamflow forecast for real‐time reservoir operation is constrained by forecast uncertainty (FU) and limited forecast horizon (FH). The effects of the two factors are complicating since increasing the FH usually provides more information for decision making in a longer time framework but with increasing uncertainty, which offsets the information gain from a longer FH. This paper illustrates the existence of an effective FH (EFH) with a given forecast, which balances the effects of the FH and FU and provides the maximum information for reservoir operation decision making. With the assumption of a concave objective function, a monotonic relationship between current operation decision and ending storage is derived. Metrics representing the error resulting from a limited forecast relative to a perfect forecast are defined to evaluate reservoir performance. Procedures to analyze the complicating effect of FU and FH and to identify EFH are proposed. Results show that: (1) when FH is short, FH is the dominating factor for determining reservoir operation, and reservoir performance exhibits a quick improvement as FH increases; (2) when FH is long, the inflow information may be too uncertain to guide reservoir operation decisions and FU becomes the dominating factor; and (3) at a medium FH, reservoir performance depends on the complicating effects of FU and FH and EFH locates with a certain balanced level of FU and FH. The statistical characteristics of EFH are illustrated with case studies with deterministic forecast and ensemble forecast. Moreover, the impacts of temporal correlation of FU, inflow variability, evaporation loss, and reservoir capacity on EFH are explored.

Deriving multiple near‐optimal solutions to deterministic reservoir operation problems

Even deterministic reservoir operation problems with a single objective function may have multiple near‐optimal solutions (MNOS) whose objective values are equal or sufficiently close to the optimum. MNOS is valuable for practical reservoir operation decisions because having a set of alternatives from which to choose allows reservoir operators to explore multiple options whereas the traditional algorithm that produces a single optimum does not offer them this flexibility. This paper presents three methods: the near‐shortest paths (NSP) method, the genetic algorithm (GA) method, and the Markov chain Monte Carlo (MCMC) method, to explore the MNOS. These methods, all of which require a long computation time, find MNOS using different approaches. To reduce the computation time, a narrower subspace, namely a near‐optimal space (NOSP, described by the maximum and minimum bounds of MNOS) is derived. By confining the MNOS search within the NOSP, the computation time of the three methods is reduced. The proposed methods are validated with a test function before they are examined with case studies of both a single reservoir (the Three Gorges Reservoir in China) and a multireservoir system (the Qing River Cascade Reservoirs in China). It is found that MNOS exists for the deterministic reservoir operation problems. When comparing the three methods, the NSP method is unsuitable for large‐scale problems but provides a benchmark to which solutions of small‐ and medium‐scale problems can be compared. The GA method can produce some MNOS but is not very efficient in terms of the computation time. Finally, the MCMC method performs best in terms of goodness‐of‐fit to the benchmark and computation time, since it yields a wide variety of MNOS based on all retained intermediate results as potential MNOS. Two case studies demonstrate that the MNOS identified in this study are useful for real‐world reservoir operation, such as the identification of important operation time periods and tradeoffs among objectives in multipurpose reservoirs.

A fuzzy-Markov-chain-based analysis method for reservoir operation

In this study, a fuzzy-Markov-chain-based stochastic dynamic programming (FM-SDP) method is developed for tackling uncertainties expressed as fuzzy sets and distributions with fuzzy probability (DFPs) in reservoir operation. The concept of DFPs used in Markov chain is presented as an extended form for expressing uncertainties including both stochastic and fuzzy characteristics. A fuzzy dominance index analysis approach is proposed for solving multiple fuzzy sets and DPFs in the proposed FM-SDP model. Solutions under a set of α-cut levels and fuzzy dominance indices can be generated by solving a series of deterministic submodels. The developed method is applied to a case study of a reservoir operation system. Solutions from FM-SDP provide a range of desired water-release policies under various system conditions for reservoir operation decision makers, reflecting dynamic and dual uncertain features of water availability simultaneously. The results indicate that the FM-SDP method could be applicable to practical problems for decision makers to obtain insight regarding the tradeoffs between economic and system reliability criteria. Willingness to obtain a lower benefit may guarantee meeting system-constraint demands; conversely, a desire to acquire a higher benefit could run into a higher risk of violating system constraints.

Effect of streamflow forecast uncertainty on real-time reservoir operation

Various hydrological forecast products have been applied to real-time reservoir operation, including deterministic streamflow forecast ( DSF), DSF-based probabilistic streamflow forecast ( pseudo-PSF, pPSF), and ensemble or probabilistic streamflow forecast (denoted as real-PSF, rPSF). DSF represents forecast uncertainty in the form of deterministic forecast errors, pPSF a conditional distribution of forecast uncertainty for a given DSF, and rPSF a probabilistic uncertainty distribution. Compared to previous studies that treat the forecast products as ad hoc inputs for reservoir operation models, this paper attempts to model the dynamic evolution of uncertainties involved in the various forecast products and explores their effect on real-time reservoir operation decisions. Through a hypothetical example of a single-objective real-time reservoir operation model, the results illustrate that forecast uncertainty exerts significant effects. Reservoir operation efficiency, as measured by a utility function, decreases as the forecast uncertainty increases but the magnitude depends on the forecast products used. In general, the utility of the reservoir operation with rPSF is nearly as high as the utility obtained with a perfect forecast. Meanwhile, the utilities of DSF and pPSF are similar to each other but not as high as rPSF. Moreover, streamflow variability and reservoir capacity can change the magnitude of the effects of forecast uncertainty, but not the relative merit of DSF, pPSF, and rPSF.

A Streamflow Forecasting Framework using Multiple Climate and Hydrological Models1

Abstract: Water resources planning and management efficacy is subject to capturing inherent uncertainties stemming from climatic and hydrological inputs and models. Streamflow forecasts, critical in reservoir operation and water allocation decision making, fundamentally contain uncertainties arising from assumed initial conditions, model structure, and modeled processes. Accounting for these propagating uncertainties remains a formidable challenge. Recent enhancements in climate forecasting skill and hydrological modeling serve as an impetus for further pursuing models and model combinations capable of delivering improved streamflow forecasts. However, little consideration has been given to methodologies that include coupling both multiple climate and multiple hydrological models, increasing the pool of streamflow forecast ensemble members and accounting for cumulative sources of uncertainty. The framework presented here proposes integration and offline coupling of global climate models (GCMs), multiple regional climate models, and numerous water balance models to improve streamflow forecasting through generation of ensemble forecasts. For demonstration purposes, the framework is imposed on the Jaguaribe basin in northeastern Brazil for a hindcast of 1974‐1996 monthly streamflow. The ECHAM 4.5 and the NCEP/MRF9 GCMs and regional models, including dynamical and statistical models, are integrated with the ABCD and Soil Moisture Accounting Procedure water balance models. Precipitation hindcasts from the GCMs are downscaled via the regional models and fed into the water balance models, producing streamflow hindcasts. Multi‐model ensemble combination techniques include pooling, linear regression weighting, and a kernel density estimator to evaluate streamflow hindcasts; the latter technique exhibits superior skill compared with any single coupled model ensemble hindcast.

A probabilistic model to support reservoir operation decisions during flash floods

A probabilistic model to assist decision makers in selecting the best reservoir operation strategy during flash floods is presented, based on Bayesian networks calibrated with the results of a rainfall—runoff model coupled with a reservoir operation model. During real-time operation, rainfall recorded in the basin is used to make probabilistic predictions of inflow discharge into the reservoir with a rainfall—runoff Bayesian network. The reservoir Bayesian network takes these probabilistic discharge values as input data and gives the probabilistic outflow discharge and water level at future time steps for the different operation strategies considered. From these probabilistic results, the best strategy for the operation of the floodgate can be selected in terms of the probability of maximum discharge downstream of the reservoir and risk of damage to the dam. Two data sets of 4000 inflow hydrographs were obtained through Monte Carlo simulation with a rainfall—runoff model and a reservoir management model. The Bayesian networks learned from the first data set and were validated with the second one. The methodology was tested successfully for one reservoir located in the south of Spain with observed data recorded during a recent flood event, checking its usefulness as a decision-making tool in real-time reservoir management.

Operational trade‐offs in reservoir control

Reservoir operation decisions require constant reevaluation in the face of conflicting objectives, varying hydrologic conditions, and frequent operational policy changes. Optimality is a relative concept very much dependent on the circumstances under which a decision is made. More than anything else, reservoir management authorities need the means to assess the impacts of various operational options. It is their responsibility to define what is desirable after a thorough evaluation of the existing circumstances. This article presents a model designed to generate operational trade‐offs common among reservoir systems. The model avoids an all‐encompassing problem formulation and distinguishes three operational modes (levels) corresponding to normal, drought, and flood operations. Each level addresses only relevant system elements and uses a static and a dynamic control module to optimize turbine performance within each planning period and temporally. The model is used for planning the operation of the Savannah River System.