Sharp convergence rates of stochastic approximation for degenerate roots

Abstract

Sharp convergence rates of stochastic approximation algorithms are given for the case where the derivative of the unknown regression function at the sought-for root is zero. The convergence rates obtained are sharp for the general step size used in the algorithms in contrast to the previous work where they are not sharp for slowly decreasing step sizes; all possible limit points are found for the case where the first matrix coefficient in the expansion of the regression function is normal; and the estimation upper bound is shown to be achieved for the multi-dimensional case in contrast to the previous work where only the one-dimensional result is proved.