Classical Stochastic Dynamic Programming Research Articles

Fuzzy Set–Based System Performance Evaluation of an Irrigation Reservoir System

This paper presents the development of fuzzy set-based performance measures for an irrigation reservoir system to answer questions on how likely a system is to fail (fuzzy reliability), how quickly it is likely to recover from failure (fuzzy resiliency), and how severe the consequences of failure are likely to be (fuzzy vulnerability). The failure/success state of the system is defined as a fuzzy event. A fuzzy set with an appropriate membership function is defined to describe both the working and failed states, to account for the system being in partly working and partly failed state. Failure of the irrigation reservoir system is related to the evapotranspiration deficit of the crops in a period. Application of the performance measures is demonstrated with the case study of the Bhadra Reservoir system in Karnataka, India. The reservoir operation is simulated using three different reservoir operating policy models: steady-state policy obtained using a fuzzy stochastic dynamic programming (FSDP) model, steady-state policy obtained using a classical stochastic dynamic programming (SDP) model, and yearly operating policy using a conceptual operating policy (COP) model. All reservoir operating policy models consider the integration of the reservoir operation policy at 10-day time periods with water allocation decisions on a daily basis by maintaining the storage continuity and soil moisture balance. Inclusion of fuzziness in the performance measure gives realistic solutions by capturing uncertainties in the reservoir operating policy models. (C) 2017 American Society of Civil Engineers.

Two Monthly Continuous Dynamic Model Based on Nash Bargaining Theory for Conflict Resolution in Reservoir System

So far many optimization models based on Nash Bargaining Theory associated with reservoir operation have been developed. Most of them have aimed to provide practical and efficient solutions for water allocation in order to alleviate conflicts among water users. These models can be discussed from two viewpoints: (i) having a discrete nature; and (ii) working on an annual basis. Although discrete dynamic game models provide appropriate reservoir operator policies, their discretization of variables increases the run time and causes dimensionality problems. In this study, two monthly based non-discrete optimization models based on the Nash Bargaining Solution are developed for a reservoir system. In the first model, based on constrained state formulation, the first and second moments (mean and variance) of the state variable (water level in the reservoir) is calculated. Using moment equations as the constraint, the long-term utility of the reservoir manager and water users are optimized. The second model is a dynamic approach structured based on continuous state Markov decision models. The corresponding solution based on the collocation method is structured for a reservoir system. In this model, the reward function is defined based on the Nash Bargaining Solution. Indeed, it is used to yield equilibrium in every proper sub-game, thereby satisfying the Markov perfect equilibrium. Both approaches are applicable for water allocation in arid and semi-arid regions. A case study was carried out at the Zayandeh-Rud river basin located in central Iran to identify the effectiveness of the presented methods. The results are compared with the results of an annual form of dynamic game, a classical stochastic dynamic programming model (e.g. Bayesian Stochastic Dynamic Programming model, BSDP), and a discrete stochastic dynamic game model (PSDNG). By comparing the results of alternative methods, it is shown that both models are capable of tackling conflict issues in water allocation in situations of water scarcity properly. Also, comparing the annual dynamic game models, the presented models result in superior results in practice. Furthermore, unlike discrete dynamic game models, the presented models can significantly reduce the runtime thereby avoiding dimensionality problems.

A Novel Solution for Stochastic Dynamic Game of Water Allocation from a Reservoir Using Collocation Method

In this study, a continuous model of stochastic dynamic game for water allocation from a reservoir system was developed. The continuous random variable of inflow in the state transition function was replaced with a discrete approximant rather than using the mean of the random variable as is done in a continuous model of deterministic dynamic game. As a result, a new solution method was used to solve the stochastic model of game based on collocation method. The collocation method was introduced as an alternative to linear-quadratic (LQ) approximation methods to resolve a dynamic model of game. The collocation method is not limited to the first and second degree approximations, compared to LQ approximation, i.e. Ricatti equations. Furthermore, in spite of LQ related problems, consideration of the stochastic nature of game on the action variables in the collocation method would be possible. The proposed solution method was applied to the real case of reservoir operation, which typically requires considering the effect of uncertainty on decision variables. The results of the solution of the stochastic model of game are compared with the results of a deterministic solution of game, a classical stochastic dynamic programming model (e.g. Bayesian Stochastic Dynamic Programming model, BSDP), and a discrete stochastic dynamic game model (PSDNG). By comparing the results of alternative methods, it is shown that the proposed solution method of stochastic dynamic game is quite capable of providing appropriate reservoir operating policies.

Incorporating Penalty Function to Reduce Spill in Stochastic Dynamic Programming Based Reservoir Operation of Hydropower Plants

AbstractThis paper proposes a framework that includes a penalty function incorporated stochastic dynamic programming (SDP) model in order to derive the operation policy of the reservoir of a hydropower plant, with an aim to reduce the amount of spill during operation of the reservoir. SDP models with various inflow process assumptions (independent and Markov‐I) are developed and executed in order to derive the reservoir operation policies for the case study of a storage type hydropower plant located in Japan. The policy thus determined consists of target storage levels (end‐of‐period storage levels) for each combination of the beginning‐of‐period storage levels and the inflow states of the current period. A penalty function is incorporated in the classical SDP model with objective function that maximizes annual energy generation through operation of the reservoir. Due to the inclusion of the penalty function, operation policy of the reservoir changes in a way that ensures reduced spill. Simulations are carried out to identify reservoir storage guide curves based on the derived operation policies. Reservoir storage guide curves for different values of the coefficient of penalty function α are plotted for a study horizon of 64 years, and the corresponding average annual spill values are compared. It is observed that, with increasing values of α, the average annual spill decreases; however, the simulated average annual energy value is marginally reduced. The average annual energy generation can be checked vis‐à‐vis the average annual spill reduction, and the optimal value of α can be identified based on the cost functions associated with energy and spill. © 2010 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

A Bayesian Stochastic Optimization Model for a Multi-Reservoir Hydropower System

This paper presents the development of an operating policy model for a multi-reservoir system for hydropower generation by addressing forecast uncertainty along with inflow uncertainty. The stochastic optimization tool adopted is the Bayesian Stochastic Dynamic Programming (BSDP), which incorporates a Bayesian approach within the classical Stochastic Dynamic Programming (SDP) formulation. The BSDP model developed in this study considers, the storages of individual reservoirs at the beginning of period t, aggregate inflow to the system during period t and forecast for aggregate inflow to the system for the next time period t + 1, as state variables. The randomness of the inflow is addressed through a posterior flow transition probability, and the uncertainty in flow forecasts is addressed through both the posterior flow transition probability and the predictive probability of forecasts. The system performance measure used in the BSDP model is the square of the deviation of the total power generated from the total firm power committed and the objective function is to minimize the expected value of the system performance measure. The model application is demonstrated through a case study of the Kalinadi Hydroelectric Project (KHEP) Stage I, in Karnataka state, India.

A stochastic dynamic programming model with fuzzy storage states for reservoir operations

Application of stochastic dynamic programming (SDP) models to reservoir optimization calls for state variables discretization. Reservoir storage volume is an important variable whose discretization has a pronounced effect on the computational efforts. The error caused by storage volume discretization is examined by considering it as a fuzzy state variable. In this approach, the point-to-point transitions between storage volumes at the beginning and end of each period are replaced by transitions between storage intervals. This is achieved by using fuzzy arithmetic operations with fuzzy numbers. In this approach, instead of aggregating single-valued crisp numbers, the membership functions of fuzzy numbers are combined. Running a simulation model with optimal release policies derived from fuzzy and non-fuzzy SDP models shows that a fuzzy SDP with a coarse discretization scheme performs as well as a classical SDP having much finer discretized space. It is believed that this advantage in the fuzzy SDP model is due to the smooth transitions between storage intervals which benefit from soft boundaries.

Comparison of Fuzzy and Nonfuzzy Optimal Reservoir Operating Policies

This paper compares reservoir operating policies obtained from fuzzy and nonfuzzy explicit stochastic dynamic programming. The reservoir operation problem for the Mansour Eddahbi dam in Morocco can be formulated as either a classical stochastic dynamic programming (SDP) problem, where the objective function stresses energy maximization with particular volumes being released for irrigation, or a fuzzy stochastic dynamic programming (FSDP) problem, in which both hydropower generation and irrigation are considered as fuzzy constraints and aggregated by the weighting method. System performance is estimated from simulation based on continuous reoptimization models using either the cost-to-go function generated by the SDP algorithm, or the membership function generated by the FSDP algorithm. Despite major differences in the mathematical representation of operating objectives and/or constraints, we show that both formulations yield similar measures of system performance.

Multireservoir System Optimization using Fuzzy Mathematical Programming

For a multireservoir system, where the number of reservoirs is large, the conventional modelling by classical stochastic dynamic programming (SDP) presents difficulty, due to the curse of dimensionality inherent in the model solution. It takes a long time to obtain a steady state policy and also it requires large amount of computer storage space, which form drawbacks in application. An attempt is made to explore the concept of fuzzy sets to provide a viable alternative in this context. The application of fuzzy set theory to water resources systems is illustrated through the formulation of a fuzzy mathematical programming model to a multireservoir system with a number of upstream parallel reservoirs, and one downstream reservoir. The study is aimed to minimize the sum of deviations of the irrigation withdrawals from their target demands, on a monthly basis, over a year. Uncertainty in reservoir inflows is considered by treating them as fuzzy sets. The model considers deterministic irrigation demands. The model is applied to a three reservoir system in the Upper Cauvery River basin, South India. The model clearly demonstrates that, use of fuzzy linear programming in multireservoir system optimization presents a potential alternative to get the steady state solution with a lot less effort than classical stochastic dynamic programming.

Bayesian stochastic optimization of reservoir operation using uncertain forecasts

Operation of reservoir systems using stochastic dynamic programming (SDP) and Bayesian decision theory (BDT) is investigated in this study. The proposed model, called Bayesian stochastic dynamic programming (BSDP), which includes inflow, storage, and forecast as state variables, describes streamflows with a discrete lag 1 Markov process, and uses BDT to incorporate new information by updating the prior probabilities to posterior probabilities, is used to generate optimal reservoir operating rules. This continuous updating can significantly reduce the effects of natural and forecast uncertainties in the model. In order to test the value of the BSDP model for generating optimal operating rules, real‐time reservoir operation simulation models are constructed using 95 years of monthly historical inflows of the Gunpowder River to Loch Raven reservoir in Maryland. The rules generated by the BSDP model are applied in an operation simulation model and their performance is compared with an alternative stochastic dynamic programming (ASDP) model and a classical stochastic dynamic programming (SDP) model. BSDP differs from the other two models in the selection of state variables and the way the transition probabilities are formed and updated.