Training of Artificial Neural Networks Using Information-Rich Data

Abstract

Artificial Neural Networks (ANNs) are classified as a data-driven technique, which implies that their learning improves as more and more training data are presented. This observation is based on the premise that a longer time series of training samples will contain more events of different types, and hence, the generalization ability of the ANN will improve. However, a longer time series need not necessarily contain more information. If there is considerable repetition of the same type of information, the ANN may not become “wiser”, and one may be just wasting computational effort and time. This study assumes that there are segments in a long time series that contain a large quantum of information. The reason behind this assumption is that the information contained in any hydrological series is not uniformly distributed, and it may be cyclic in nature. If an ANN is trained using these segments rather than the whole series, the training would be the same or better based on the information contained in the series. A pre-processing can be used to select information-rich data for training. However, most of the conventional pre-processing methods do not perform well due to large variation in magnitude, scale and many zeros in the data series. Therefore, it is not very easy to identify these information-rich segments in long time series with large variation in magnitude and many zeros. In this study, the data depth function was used as a tool for the identification of critical (information) segments in a time series, which does not depend on large variation in magnitude, scale or the presence of many zeros in data. Data from two gauging sites were used to compare the performance of ANN trained on the whole data set and just the data from critical events. Selection of data for critical events was done by two methods, using the depth function (identification of critical events (ICE) algorithm) and using random selection. Inter-comparison of the performance of the ANNs trained using the complete data sets and the pruned data sets shows that the ANN trained using the data from critical events, i.e., information-rich data (whose length could be one third to half of the series), gave similar results as the ANN trained using the complete data set. However, if the data set is pruned randomly, the performance of the ANN degrades significantly. The concept of this paper may be very useful for training data-driven models where the training time series is incomplete.