Design of hybrid neural networks of the ensemble structure

Victor Sineglazov,Anatoly Kot

doi:10.15587/1729-4061.2021.225301

Abstract

This paper considers the structural-parametric synthesis (SPS) of neural networks (NNs) of deep learning, in particular convolutional neural networks (CNNs), which are used in image processing. It has been shown that modern neural networks may possess a variety of topologies. That is ensured by using unique blocks that determine their essential features, namely, the compression and excitation unit, the attention module convolution unit, the channel attention module, the spatial attention module, the residual unit, the ResNeXt block. This, first of all, is due to the need to increase their efficiency in the processing of images. Due to the large architectural space of parameters, including the type of unique block, the location in the structure of the convolutional neural network, its connections with other blocks, layers, computing costs grow nonlinearly. To minimize computational costs while maintaining the specified accuracy this work set tasks of both the generation of possible topology and structural-parametric synthesis of convolutional neural networks. To resolve them, the use of a genetic algorithm (GA) has been proposed. Parameter configuration was implemented using a genetic algorithm and modern gradient methods (GM). For example, stochastic gradient descent with momentum, accelerated Nesterov gradient, adaptive gradient algorithm, distribution of the root of the mean square of the gradient, assessment of adaptive momentum, adaptive Nesterov momentum. It is assumed to use such networks in the intelligent medical diagnostic system (IMDS), for determining the activity of tuberculosis. To improve the accuracy of solving the classification problem in the processing of images, the ensemble structure of hybrid convolutional neural networks (HCNNs) has been proposed in the current work. The parallel structure of the ensemble with the merged layer was used. Algorithms of optimal choice and integration of features in the construction of the ensemble have been developed

Highlights

IntroductionOne of the promising directions of the modern stage of health informatization is the development of intelligent medical diagnostic systems that provide support for decision-making by a doctor
Healthcare information support systems are actively developing
The intelligent element of IMDS is the neural network used both for the image processing from ultrasound studies (USS), computed tomography (CT), magnetic resonance imaging (MRI) studies and to support decision-making regarding the final diagnosis

Summary

Introduction

One of the promising directions of the modern stage of health informatization is the development of intelligent medical diagnostic systems that provide support for decision-making by a doctor. This is primarily due to the lack of sufficient experience from doctors, the rapid development of medicine, and the lack of time resources for improving the skills and experience of staff. Hybrid neural networks (HNNs) consist of different structures united in the interest of achieving the goals based on deep learning This makes it possible to solve complex problems, first of all, the processing of medical images, which cannot be solved on the basis of individual methods and technologies. A convolutional neural network is built on the basis of a convolution operation, which makes it possible to train CNN on certain parts of the image, iteratively increasing the local learning area of a separate convolutional nucleus

Objectives

Methods

Findings

Conclusion