Automation of the process of selecting hyperparameters for artificial neural networks for processing retrospective text information

Abstract

Neural network technologies are successfully used in solving problems from various areas of the economy - industry, agriculture, medicine. The problems of substantiating the choice of architecture and hyperparameters of artificial neural networks (ins) aimed at solving various classes of applied problems are caused by the need to improve the quality and speed of deep ins training. Various methods of optimizing ins hyperparameters are known, for example, using genetic algorithms, but this requires writing additional software. To optimize the process of selecting hyperparameters, Google research has developed the KerasTuner Toolkit, which is a user-friendly platform for automated search for optimal hyperparameter combinations. In the described Kerastuner Toolkit, you can use random search, Bayesian optimization, or Hyperband methods. In numerical experiments, 14 hyperparameters varied: the number of blocks of convolutional layers and their forming filters, the type of activation functions, the parameters of the «dropout» regulatory layers, and others. The studied tools demonstrated high optimization efficiency while simultaneously varying more than a dozen parameters of the convolutional network, while the calculation time on the Colaboratory platform for the studied INM architectures was several hours, even with the use of GPU graphics accelerators. For ins focused on processing and recognizing text information in natural language (NLP), the recognition quality has been improved to 83-92%.