Self-Organizing Optimization Based on Caputo’s Fractional Order Gradients

Abstract

This paper analyses the condition necessary to guarantee no divergence for Caputo’s fractional order gradient descent (C-FOG) algorithm on multivariate functions. C-FOG is self-organizing, computationally efficient, simple, and understandable. It converges faster than the classical gradient-based optimization algorithms and converges to slightly different points when the order of the fractional derivative is different. The additional freedom of the order is very useful in situations where the diversity of convergence is required, and it also allows for more precise convergence. Comparative experiments on a typical poor conditioned function and adversarial sample generation frameworks demonstrate the convergence performance of C-FOG, showing that it outperforms currently popular algorithms in terms of convergence speed, and more excitingly, the diversity of convergence allows it to exhibit stronger and more stable attack capability in adversarial sample generation procedures (The code for experiments is available at: https://github.com/mulertan/self_optimizing/tree/main, accessed on 30 April 2024).