Hydrological Big Data Prediction Based on Similarity Search and Improved BP Neural Network

Abstract

Large amount of hydrological data set is a kind of big data, which has much hidden and potentially useful knowledge. Hydrological prediction is important for the state flood control and drought relief. How to forecast accurately and timely with hydrological big data becomes a big challenge. There are some forecasting techniques used widely. However, they are limited by their adaptability, the data volume and the data feature. The most important problems are the high time consumption, low accuracy and bad adaptability of prediction. In this paper, a new forecasting approach based on an integration of two tasks of data mining is put forward. This approach which is called S LMDBP combines similarity search and Levenberg-Marquardt(LM) algorithm improved Double-hidden layer Back Propagation(BP) neural network. A specialized data pretreatment including three parts is applied to process the hydrological data. The results of similarity search are then input into the improved BP neural network, which not only reduces dimensionality of training data without losing important patterns, but also improves the accuracy of the prediction. A set of experiments are conducted to validate the proposed approach. The data in the experiment is the daily water level data of three stations in Jiangxi province of China from 1950 to 2010. The experimental results demonstrate the real-timing, accuracy and robustness of our approach.