DEVELOPMENT OF THE METHOD OF UNSUPERVISED TRAINING OF CONVOLUTIONAL NEURAL NETWORKS BASED ON NEURAL GAS MODIFICATION

Viacheslav Moskalenko

doi:10.21303/2585-6847.2017.00469

Abstract

Technologies for computer analysis of visual information based on convolutional neural networks have been widely used, but there is still a shortage of working algorithms for continuous unsupervised training and re-training of neural networks in real time, limiting the effectiveness of their functioning under conditions of nonstationarity and a priori uncertainty. In addition, the back propagation method for learning multi-layer neural networks requires significant computational resources and the amount of marked learning data, which makes it difficult to implement them in autonomous systems with limited resources. One approach to reducing the computational complexity of deep machine learning and overfitting is use of the neural gas principles to implement learning in the process of direct information propagation and sparse coding to increase the compactness and informativeness of feature representation. The paper considers the use of sparse coding neural gas for learning ten layers of the VGG-16 neural network on selective data from the ImageNet database. At the same time, it is suggested that the evaluation of the effectiveness of the feature extractor learning be carried out according to the results of so-called information-extreme machine learning with the teacher of the output classifier. Information-extreme learning is based on the principles of population optimization methods for binary coding of observations and the construction of radial-basic decision rules optimal in the information criterion in the binary Hamming space. According to the results of physical modeling, it is shown that learning without a teacher ensures the accuracy of decision rules to 96.4 %, which is inferior to the accuracy of learning with the teacher, which is equal to 98.7 %. However, the absence of an error in the training algorithm for the backward propagation of the error causes the prospect of further research towards the development of meta-optimization algorithms to refine the feature extractor's filters and parameters of the unsupervised training algorithm

Highlights

DEVELOPMENT OF THE METHOD OFIt is proposed to use the Technologies of computer analysis of visual information have received wide application in robotic systems and infocommunicationUNSUPERVISED TRAINING OF CONVOLUTIONAL NEURAL NETWORKS BASEDON NEURAL GAS MODIFICATION principles of neural gas and sparse coding for the training of the hierarchical extractor of visual features using the examservices for various purposes
The proposed unsupervised machine learning algorithm of the multi-layered convolutional neural network VGG-16 is used to synthesize the extractor of the feature description and classifier of objects from the alphabet {Xok | k = 1, K}, where K=21
The graphs of the change in the maxima of the information criterion (1) during the process of optimizing the field of the control tolerances for the value of the features formed by the 10 layers of the extractor learned without a teacher by the modified neural gas method are shown in Fig. 2, a

Summary

Introduction

DEVELOPMENT OF THE METHOD OFIt is proposed to use the Technologies of computer analysis of visual information have received wide application in robotic systems and infocommunicationUNSUPERVISED TRAINING OF CONVOLUTIONAL NEURAL NETWORKS BASEDON NEURAL GAS MODIFICATION principles of neural gas and sparse coding for the training of the hierarchical extractor of visual features using the examservices for various purposes. UNSUPERVISED TRAINING OF CONVOLUTIONAL NEURAL NETWORKS BASED. Technologies for computer analysis of visual information based on convolutional neural networks have been widely used, but there is still a shortage of working algorithms for continuous unsupervised training and re-training of neural networks in real time, limiting the effectiveness of their functioning under conditions of nonstationarity and a priori uncertainty. The back propagation method for learning multi-layer neural networks requires significant computational resources and the amount of marked learning data, which makes it difficult to implement them in autonomous systems with limited resources. One approach to reducing the computational complexity of deep machine learning and overfitting is use of the neural gas principles to implement learning in the process of direct information propagation and sparse coding to increase the compactness and informativeness of feature representation

Methods

Results

Conclusion