Generalization of the gradient method with fractional order gradient direction

Abstract

Fractional calculus is an efficient tool, which has the potential to improve the performance of gradient methods. However, when the first order gradient direction is generalized by fractional order gradient one, the corresponding algorithm converges to the fractional extreme point of the target function which is not equal to the real extreme point. This drawback critically hampers the application of this method. To solve such a convergence problem, the current paper analyzes the specific reasons and proposes three possible solutions. Considering the long memory characteristics of fractional derivative, short memory principle is a prior choice. Apart from the truncation of memory length, two new methods are developed to reach the convergence. The former is the truncation of the infinite series, and the latter is the modification of the constant fractional order. Finally, six illustrative examples are performed to illustrate the effectiveness and practicability of proposed methods.