The gradient descent method from the perspective of fractional calculus

Abstract

Motivated by gradient methods in optimization theory, we give methods based on ψ‐fractional derivatives of order α in order to solve unconstrained optimization problems. The convergence of these methods is analyzed in detail. This paper also presents an Adams–Bashforth–Moulton (ABM) method for the estimation of solutions to equations involving ψ‐fractional derivatives. Numerical examples using the ABM method show that the fractional order α and weight ψ are tunable parameters, which can be helpful for improving the performance of gradient descent methods.