Efficient mini-batch stochastic gradient descent with Centroidal Voronoi Tessellation for PDE-constrained optimization under uncertainty

Abstract

The study of optimal control problems under uncertainty plays an important role in scientific numerical simulations. This class of optimization problems is frequently utilized in engineering, biology and finance. Although stochastic gradient descent is a well-known stochastic optimization technique for solving the approximate optimal control problem, it often exhibits a slow convergence rate due to the inherent variance in gradient approximation. To address this issue, we propose a more efficient stochastic optimization algorithm based on Centroidal Voronoi Tessellation sampling. This strategy reduces the error in gradient estimation compared to the conventional mini-batch stochastic gradient descent method. Our approach involves dividing the whole snapshot set into several Voronoi cells with low variance and extracting samples with good uniformity from each region to construct an unbiased estimation of the full gradient. Our method can economically and stably approximate numerical optimal control functions even with a fixed step size. Numerical results demonstrate that the proposed method is a reliable algorithm with great potential for applications in the field of optimization.