Hydrological Data Processing Research Articles

Potential Uses of Stormwater Runoff Sedimentgs Retained in a Constructed-Wetland/Storage-Tank

Objective: The goal of this work was to establish the quality conditions of the sediments retained in a constructed-wetland/storage-tank, which are to be used as productive material. Materials and methods: Sediments were collected every fortnight for five months in 2016, with sediment traps specially designed for the study case, to analyze particle size distribution, total organic carbon, and heavy metals concentrations. Sediment and hydrological data processing were performed using principal component analysis and multiple correspondence analysis. Results: The maximum measured concentrations were 110.80 mg/kg, 263.25 mg/kg, 798.85 mg/kg, and 3067 mg/kg for Cr, Cu, Pb, and Zn respectively; for total organic carbon was 20.6 mg. The sediment particles were considered thin (D50 < 150μm). Interesting relationships between rainfall and sediment characteristics were found. In shorter dry seasons, higher particle diameter, lower concentrations of heavy metals, and higher concentrations of total organic carbon were seen. Conclusions: Sediments are suitable for use in activities such as land remediation, vegetation of soils that are not for agricultural use, and as materials for ornamental and recreational areas. It is possible to use the sediments in embankments as tolerable soil for fillings of concrete structures and culverts in the core or foundation area.

Do ARMA Models Provide Better Gap Filling in Time Series of Soil Temperature and Soil Moisture? The Case of Arable Land in the Kulunda Steppe, Russia

The adoption of climate-smart agriculture requires the comprehensive development of environmental monitoring tools, including online observation of climate and soil settings. They are often designed to measure soil properties automatically at different depths at hour or minute intervals. It is essential to have a complete dataset to use statistical models for the prediction of soil properties and to make short-term decisions regarding soil tillage operations and irrigation during a vegetation period. This is also important in applied hydrological studies. Nevertheless, the time series of soil hydrological measurements often have data gaps for different reasons. The study focused on solving a problem of gap-filling in hourly time series of soil temperature and moisture, measured at the 30 cm depth using a weighted gravitation lysimeter station while meteorological data were recorded simultaneously by a weather station. The equipment was installed in the Kulunda Steppe in the Altai Krai, Russia. Considering that climate conditions affect soil temperature and moisture content directly, we did a comparative analysis of the gap-filling performance using the three imputation methods—linear interpolation, multiple linear regression, and extended ARMA (p,q) models with exogenous climatic variables. The results showed that, according to the minimum of the mean absolute error, ARMA (p,q) models with optimally selected order parameters, and an adaptive window, had some advantages compared to other single-imputation methods. The ARMA (p,q) model produced a good quality of gap-filling in time series with the mean absolute error of 0.19 °C and 0.08 Vol. % for soil temperature and moisture content, respectively. The findings supplemented the methodology of hydrological data processing and the development of digital tools for the online monitoring of climate and soil properties in agriculture.

2D singular spectrum analysis for hydrological data processing

Singular spectrum analysis (SSA), among other methods of decomposition based on empirical orthogonal functions (EOF), expands opportunities for statistical analysis of data fields. The authors suggest using two-dimensional version of SSA (2D-SSA) for processing in-situ hydrological data and prove that this method is applicable for this purpose. The authors consider key features and advantages of this approach as well as its procedural sequence and offer physical interpretation of the results based on full understanding of contribution of various factors to the hydrological situation. The examples of processing of the field measurements carried out in the shelf zone of the Sea of Japan are presented in the paper. The first dataset contained the data of profiling for one-and-a-half-hours with the intervals of two minutes at one observation point. The second consisted of the data of a single profile section along which observations were made three times several weeks apart. In both cases, the singled out two-dimensional structures were ordered according to the degree of their influence on the general hydrological picture. The filtration carried out this way made it possible in the first case to consider wave dynamics in detail, and in the second case to clarify the features of spatial geometry of the currents and vertical convection. In general, the use of 2D-SSA made it possible to interpret the results of the survey with a high degree of accuracy.

HydroDS: Data services in support of physically based, distributed hydrological models

Physically based distributed hydrologic models require geospatial and time-series data that take considerable time and effort in processing them into model inputs. Tools that automate and speed up input processing facilitate the application of these models. In this study, we developed a set of web-based data services called HydroDS to provide hydrologic data processing ‘software as a service.’ HydroDS provides functions for processing watershed, terrain, canopy, climate, and soil data. The services are accessed through a Python client library that facilitates developing simple but effective data processing workflows with Python. Evaluations of HydroDS by setting up the Utah Energy Balance and TOPNET models for multiple headwater watersheds in the Colorado River basin show that HydroDS reduces the input preparation time compared to manual processing. It also removes the requirements for software installation and maintenance by the user, and the Python workflows enhance reproducibility of hydrologic data processing and tracking of provenance.

Prediction of Marine Pycnocline Based on Kernel Support Vector Machine and Convex Optimization Technology.

With the explosive growth of ocean data, it is of great significance to use ocean observation data to analyze ocean pycnocline data in military field. However, due to natural factors, most of the time the ocean hydrological data is not complete. In this case, predicting the ocean hydrological data by partial data has become a hot spot in marine science. In this paper, based on the traditional statistical analysis literature, we propose a machine-learning ocean hydrological data processing process under big data. At the same time, based on the traditional pycnocline gradient determination method, the open Argo data set is analyzed, and the local characteristics of pycnocline are verified from several aspects combined with the current research about pycnocline. Most importantly, in this paper, the combination of kernel function and support vector machine(SVM) is extended to nonlinear learning by using the idea of machine learning and convex optimization technology. Based on this, the known pycnocline training set is trained, and an accurate model is obtained to predict the pycnocline in unknown domains. In the specific steps, this paper combines the classification problem with the regression problem, and determines the proportion of training set and test formula set by polynomial regression. Subsequently, the feature scaling of the input data accelerated the gradient convergence, and a grid search algorithm with variable step size was proposed to determine the super parameter c and gamma of the SVM model. The prediction results not only used the confusion matrix to analyze the accuracy of GridSearch-SVM with variable step size, but also compared the traditional SVM and the similar algorithm. At the end of the experiment, two features which have the greatest influence on the Marine density thermocline are found out by the feature ranking algorithm based on learning.

Improving a stochastic multi-site generation model of daily rainfall using discrete wavelet de-noising: a case study to a semi-arid region

This paper presents a combined model of wavelet transform analysis and multi-site stochastic generation model of daily rainfall data. The followed methodology demonstrates the feasibility of the discrete wavelets transform for de-noising process as part of hydrological data processing. The classical two-part stochastic model has been investigated at the basin scale to simulate multi-site daily rainfall time series, which is among the most useful information used in water resources management. However, despite the importance of such information, as far as we know, no previous contribution has been conducted to tackle the issue for Algerian watershed. Such requirement has spurred the authors to investigate the performance of a multi-site stochastic process in this study area. Therefore, to develop this investigation, a modified stochastic model based on Wilks approach was adopted. According to the literature, the adopted model led to good results in solving the adverse effect of random noise problem. Overall, the validation of the statistical characteristics of the obtained generated series demonstrates that the model performs very well with noisy data as well as with the de-noised ones. Furthermore, the use of the pre-treated daily rainfall data in the stochastic multi-site model has contributed to improve substantially the spatial dependency results for the semi-arid study area.

A review on missing hydrological data processing

Like almost all fields of science, hydrology has benefited to a large extent from the tremendous improvements in scientific instruments that are able to collect long-time data series and an increase in available computational power and storage capabilities over the last decades. Many model applications and statistical analyses (e.g., extreme value analysis) are based on these time series. Consequently, the quality and the completeness of these time series are essential. Preprocessing of raw data sets by filling data gaps is thus a necessary procedure. Several interpolation techniques with different complexity are available ranging from rather simple to extremely challenging approaches. In this paper, various imputation methods available to the hydrological researchers are reviewed with regard to their suitability for filling gaps in the context of solving hydrological questions. The methodological approaches include arithmetic mean imputation, principal component analysis, regression-based methods and multiple imputation methods. In particular, autoregressive conditional heteroscedasticity (ARCH) models which originate from finance and econometrics will be discussed regarding their applicability to data series characterized by non-constant volatility and heteroscedasticity in hydrological contexts. The review shows that methodological advances driven by other fields of research bear relevance for a more intensive use of these methods in hydrology. Up to now, the hydrological community has paid little attention to the imputation ability of time series models in general and ARCH models in particular.

Analysis of Phenomenels with Hydrologic Large Risk in the Hydrographic Basin of the Trotuş River

Abstract The paper presents an analysis of the high hydrological risk phenomena formed in the hydrographic basin of the Trotuş River in the last period of time. The Trotuş River and the tributaries are monitored by 21 hydrometric stations. Precipitation volume processing indicated a number of risk factors that have prevailed over the last 20 years. The hydrological data processing revealed the presence of several flood flows in the same year. The effects of the floods have materialized through the excessive degradation of river bedside regulation and shore defence works. The floods of the past 25 years have resulted in the destruction of a large number of economic and social objectives in the Trotuş River area, as well as human losses. Parameters of hydroclimatic risk highlighted by research are represented by torrential precipitations, floods with high probability, high frequency of high-flow flows, formation of high erosion velocities of the bed, etc. Parameters of hydroclimatic risk impose special conditions for the design of river regularization and shore defence.

BUILDING A COMPLETE FREE AND OPEN SOURCE GIS INFRASTRUCTURE FOR HYDROLOGICAL COMPUTING AND DATA PUBLICATION USING GIS.LAB AND GISQUICK PLATFORMS

Abstract. Building a complete free and open source GIS computing and data publication platform can be a relatively easy task. This paper describes an automated deployment of such platform using two open source software projects – GIS.lab and Gisquick. GIS.lab (http: //web.gislab.io) is a project for rapid deployment of a complete, centrally managed and horizontally scalable GIS infrastructure in the local area network, data center or cloud. It provides a comprehensive set of free geospatial software seamlessly integrated into one, easy-to-use system. A platform for GIS computing (in our case demonstrated on hydrological data processing) requires core components as a geoprocessing server, map server, and a computation engine as eg. GRASS GIS, SAGA, or other similar GIS software. All these components can be rapidly, and automatically deployed by GIS.lab platform. In our demonstrated solution PyWPS is used for serving WPS processes built on the top of GRASS GIS computation platform. GIS.lab can be easily extended by other components running in Docker containers. This approach is shown on Gisquick seamless integration. Gisquick (http://gisquick.org) is an open source platform for publishing geospatial data in the sense of rapid sharing of QGIS projects on the web. The platform consists of QGIS plugin, Django-based server application, QGIS server, and web/mobile clients. In this paper is shown how to easily deploy complete open source GIS infrastructure allowing all required operations as data preparation on desktop, data sharing, and geospatial computation as the service. It also includes data publication in the sense of OGC Web Services and importantly also as interactive web mapping applications.

Server‐side workflow execution using data grid technology for reproducible analyses of data‐intensive hydrologic systems

AbstractMany geoscience disciplines utilize complex computational models for advancing understanding and sustainable management of Earth systems. Executing such models and their associated data preprocessing and postprocessing routines can be challenging for a number of reasons including (1) accessing and preprocessing the large volume and variety of data required by the model, (2) postprocessing large data collections generated by the model, and (3) orchestrating data processing tools, each with unique software dependencies, into workflows that can be easily reproduced and reused. To address these challenges, the work reported in this paper leverages the Workflow Structured Object functionality of the Integrated Rule‐Oriented Data System and demonstrates how it can be used to access distributed data, encapsulate hydrologic data processing as workflows, and federate with other community‐driven cyberinfrastructure systems. The approach is demonstrated for a study investigating the impact of drought on populations in the Carolinas region of the United States. The analysis leverages computational modeling along with data from the Terra Populus project and data management and publication services provided by the Sustainable Environment‐Actionable Data project. The work is part of a larger effort under the DataNet Federation Consortium project that aims to demonstrate data and computational interoperability across cyberinfrastructure developed independently by scientific communities.

A Hydrometric and Hydrological Approach Test at Microscale

The objectives of this article were: to test the hydrometric accuracy of some water level variation measuring and recording devices using a metal measuring tank with weir, and to automatically determine runoff intensities and elements. These field tests were performed in order to ensure high accuracy and low uncertainty of studies at hydrological micro-scale (plot scale). Hydrometric tests targeted two level measurement conditions: (i) rise and storage - without overflowing, volumetric measurement V= f (H); and (ii) rise and overflowing, weir measurement Q = f (H). Hydrometric accuracy was evaluated by comparing the measured and recorded level using three instruments with a tell-tale level. This field experiment was conducted in Voineşti Experimental Basin, belonging to the National Institute of Hydrology and Water Management. Levels series data were processed with the software application ParExp v1, in order to automatically convert them into discharges (Q). Hydrometric and hydrological test results highlighted certain aspects. The accuracy estimated for water fluctuation measurement and recording instruments in a weir water tank, for both level measurement conditions revealed accuracy errors (insufficient accuracy) when the runoff hydraulics was changed (storage Qacum ÷ overflowing Qdev). To remedy such instrumental deficiencies, a metrological control shall be performed under specific operational conditions (e.g., water tank) in order to meet increasing needs for high quality hydrological data. The hydrological data processing using the ParExp v1 software application, at the junction of specific “rise and storage” and “rise and overflowing” flows indicated a temporal error (delay). The user may remedy this error by eliminating data from the Qacum-Qdev junction range until reaching the maximum/stabilized level.Finally, we estimate that through the improvement of some technical elements, hydrological data obtained at a micro-scale level can be used for hydrological models of calibration.

应用马尔文激光粒度仪分析泥沙数据处理模板

马尔文激光粒度分布仪始于本世纪初应用于河流泥沙粒度分析，虽然与传统的沉降法分析原理有着本质的区别，但其仪器所具备的操作简便、分析数据稳定和处理自动化程度高、成果获取快等优点，深受使用者的青睐，目前已在河流泥沙颗粒分析中得到了应用和普及。然而，用户在仪器使用中会遇到一些具体问题：如基础参数设定和检查不方便、测试数据的好坏无法自动检测和提示处理，测试的颗粒级配数据特征值及极值挑选比较麻烦、分析报告格式不统一、数据读取和输出处理繁琐等。为解决这些问题，并统一各泥沙监测站的激光仪操作方法和要求，在充分利用激光仪现有功能的基础上，开发研制出了一套“马尔文激光粒度仪泥沙数据处理模板”，该模板能自动配置基础参数、对分析数据进行质量检查并提示，按测站编码打印标准格式报告单、形成批量分析数据的文本输出文件等，可直接提供给南方片的整汇编软件进行整理、整编等。经过近几年的运行取得良好效果，提高了工作效率，减少了出错机率，保障了成果质量。 Malvern laser particle size analyzer has been applied in the particle analysis of river sediments since the beginning of this century, and has been highly praised by users for simple operation, stable data from analysis, high automation of processing and quick achievements, etc. regardless of its significant deviation from the principles of traditional sedimentation analysis approach. Now, it has been applied and popularized in the particle analysis of river sediments. Nevertheless, users may run into some problems in the use of this equipment, such as, inconvenient setting and check of fundamental parameters, lack of automatic detection and warning about quality of test data, troublesome selection of characteristic and extreme values in the data of tested particle grading, inconsistent form of analysis report, and cumber-some reading and output of data, etc. To solve these problems and unify the principles and requirements for operation of laser analyzer at the sediment monitoring stations, a “sediment data processing template for Malvern laser particle size analyzer” is developed while making full use of the existing functions of laser analyzer. This template is able to configure the fundamental parameters automatically, check the quality of analytic data and give warnings, print the reports in standard form according to the code of each monitoring station, and generate the textual output of analytic data in batches, etc., which can be directly sorted out and edited in the Southern Hydrological Data Processing (SHDP) software. The operation for several years has achieved favorable effects, improved working efficiency, reduced error rate and ensured the quality of achievements.

水文资料整编系统在恢复刊印水文年鉴中的应用

在我国水文年鉴刊印史上，从1990年代起各省有十几年不等的水文资料没有刊印，为此，在保证水文年鉴刊印质量的前提下，如何对这些水文资料及时地进行恢复刊印，具有重大的现实意义；长江水利委员会水文局在2012年底，应用水文资料整编系统恢复刊印了1991年度的两册水文年鉴资料，积累了很多经验。本文针对水文资料整编系统的应用，说明其在恢复刊印水文年鉴应该把握的问题，以期为该系统的应用提供参考。 The South-Region Hydrologic Data Processing Software (SHDP) is a multi-functional data processing platform that integrates the processing/compilation of hydrologic data, information management, as well as result output as a whole. Its backstage server is the Hydrological Database (Version 4.0). The SHDP software has been so far successfully applied in hydrological departments across more than twenty provinces, cities, and autonomous regions. A problem present in the hydrologic yearbook publishing history of China is that, since the 1990s, many provincial hydrological yearbooks have not been published for dozens of years. Therefore, it is of significant practical importance to timely restore the printing of these missing materials while guaranteeing their quality. In response, the Hydrology Bureau of Changjiang Water Resources Commission has systematically restored the data of two yearbooks for 1991 using the SHDP software in 2012. In this paper, instructions for using the SHDP and the process of realizing the restoring of data in hydrologic yearbook publishing are both presented.