Training Artificial Neural Networks by Generalized Likelihood Ratio Method: An Effective Way to Improve Robustness

Abstract

In this work, we proposed a generalized likelihood ratio method capable of training the artificial neural networks with more flexibility: (a)training with discrete activation and loss functions, while the traditional back propagation method cannot train the artificial neural networks with such activations and loss; (b)involving neuronal noises during training and prediction, which will improve the freedom of the model and make it more adaptable to the real environment, especially when environmental noises exist. Numerical results show that the robustness of various artificial neural networks trained by the new method is significantly improved when the input data is affected by both the natural noises and adversarial attacks.