Increasing the effectiveness of intelligent module by enlarging training dataset from real data

Abstract

This paper concentrates on the design of the intelligent module and neural networks in the area of intelligent data processing. It raises the following issues: what real problems are faced in training datasets for neural networks, how they can be solved, and how an intelligent module can be useful in this area. During the development of a neural network system, it is important to improve the network performance after the system’s release. Additional training data need to be taken from real-life data, combined with existing training data to produce a better version of the neural network’s weights. Since the number of these samples can be large, it’s useful to filter out samples similar to those already included in training data. We designed a module architecture and an algorithm that employ cascading filters in order to find the best samples from real data for the training dataset. The key feature of the intelligent module is that it does not generate a completely new dataset, but uses saved data from the real-life samples and samples from the existing database. Weights are then updated during the training of the neural network and then filtered using a special algorithm.