A probabilistic Markov model of reservoir state transitions is developed which, in combination with a simulation model, offers a computational tool for estimating response surface gradients of water resources planning problems. Use of the gradient estimation capability provides a substantial improvement in efficiency of the search for improved water resources configurations. The estimation capability arises by recognizing and using the reservoir state transition changes that would result from a modification of decision variables. The procedure is highly flexible in that it functions under such complicated situations as multiple‐objective environments, nonlinear benefit loss functions, and complex connectivity of river basin components.