Huber Loss Function Based on Variable Step Beetle Antennae Search Algorithm with Gaussian Direction

Abstract

Huber Loss function is a regression type loss function, mainly used for optimization problems.In order to make the predicted value close to the true value and minimize the distance between them, the traditional loss function generally uses the gradient descent method to find the minimum value. However, the gradient descent method takes too long and consumes a lot of memory. It may not necessarily find the global optimal value, and it is easy to fall into the problem of local optimization. The data of the unconstrained function is complicated and huge, and the Huber Loss function sets the hyper-parameter δ to optimize MSE and MAE, we must iterate continuously to train the hyper-parameters to obtain more accurate parameter values, which will cause a problem of large amount of calculations. The beetle antennae search algorithm is a simple biological intelligence optimization algorithm that can provide the best solution based on huge data, with fast convergence speed, small amount of calculation, simple operation, and no need to know the specific function. In order to improve the accuracy of finding the global optimal value, this paper changed the traditional fixed-step antennae search algorithm to variable-step beetle antennae search algorithm. This article will discuss the application of the gradient descent method in the regression loss function, and then use the variable step size beetle antennae search algorithm to optimize Huber Loss function.