AbstractThis study covers the output‐feedback model predictive control (MPC) of nonlinear systems subjected to stochastic disturbances and state chance constraints. The stochastic optimal control problem is solved in a stochastic dynamic programming fashion, and the output‐feedback control is performed with the extended Kalman filter. The information state is summarized as a dynamic Gaussian belief model. Thus, the stochastic Bellman equation is transformed into a deterministic equation using this model. The resulting constrained Bellman equation is solved using the proposed constrained, approximate dynamic programming algorithm. The algorithm is proved to have a Q‐superlinear local convergence rate. Numerical experiments show that the proposed algorithm can attain good control performance and reasonable chance‐constraint satisfaction and is computationally efficient owing to its dynamic programming structure.