Automation of Architecture Justification and Parameters Selection of Artificial Neural Networks for Intelligent Detection of Cyber-Physical Threats

Abstract

The problems of improving the quality of training of deep artificial neural networks (ANN) for various applied tasks require automatization of the selection of hyperparameters of neural networks. The KerasTuner software toolkit can be used to automate the search for optimal values of ANN hyperparameters. It includes random search methods, Bayesian optimization, etc. The formation of training text samples for neural network identification of cyber-physical threats is a separate scientific and methodological task. The complexity of the problem is due to the diversity of the ontology of the key terms of the cyberphysical thesaurus, the variety of styles of lexicological content, as well as the partial intersection of the content of previously identified ontological categories. In the process of experimental study of hyperparameters of deep ANNs being developed, models of “embedding”, “bag of words” and dense vector representation in Python were compared. On the basis of a systematic approach, an information-morphological matrix of thematic blocks is constructed. In the conducted experiments, the values of parameters such as the number of convolutional blocks, the number of their filters, the type of activation functions, the parameters of the “dropout” layers, etc. were changed. The studied tools provided optimization of hyperparameters of the convolutional network, while the calculation time on the Colaboratory platform for the studied ANN architectures using GPU graphics accelerators was 5…9 o’clock. The developed modified algorithm for computer detection of cyberphysical threats in electronic resources allowed to substantiate alternative architectures and optimize the main hyperparameters of ANN.