Abstract
The classical Method of Successive Approximations (MSA) is an iterative method for solving stochastic control problems and is derived from Pontryagin’s optimality principle. It is known that the MSA may fail to converge. Using careful estimates for the backward stochastic differential equation (BSDE) this paper suggests a modification to the MSA algorithm. This modified MSA is shown to converge for general stochastic control problems with control in both the drift and diffusion coefficients. Under some additional assumptions the rate of convergence is shown. The results are valid without restrictions on the time horizon of the control problem, in contrast to iterative methods based on the theory of forward-backward stochastic differential equations.
Highlights
Stochastic control problems appear naturally in a range of applications in engineering, economics and finance
We consider a modification to the method of successive approximations (MSA), see Algorithm 1
In Theorem 2.5 we show that the modified method of successive approximations, converges for arbitrary T, and in Corollary 2.6, we show logarithmic convergence rate for certain stochastic control problems
Summary
Stochastic control problems appear naturally in a range of applications in engineering, economics and finance. Given the augmented Hamiltonian, let us introduce the modified MSA in Algorithm 1 which consists of successive integrations of the state and adjoint systems and updates to the control. We need to modify the algorithm in such way so that we ensure convergence With this in mind the desirability of the the augmented Hamiltonian (6) for updating the control becomes clear, as long as it still characterises optimal controls like H does. In Theorem 2.5 we show that the modified method of successive approximations, converges for arbitrary T , and in Corollary 2.6, we show logarithmic convergence rate for certain stochastic control problems. The main contributions of this paper are the probabilistic proof of convergence of the modified method of successive approximations and establishing convergence rate for a specific type of optimal control problems. In Appendix 1 we recall an auxiliary lemma which is needed in the proof of Corollary 2.6
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
