Gaps In Time Series Research Articles

Ecosystem Resilience Monitoring and Early Warning Using Earth Observation Data: Challenges and Outlook

AbstractAs the Earth system is exposed to large anthropogenic interferences, it becomes ever more important to assess the resilience of natural systems, i.e., their ability to recover from natural and human-induced perturbations. Several, often related, measures of resilience have been proposed and applied to modeled and observed data, often by different scientific communities. Focusing on terrestrial ecosystems as a key component of the Earth system, we review methods that can detect large perturbations (temporary excursions from a reference state as well as abrupt shifts to a new reference state) in spatio-temporal datasets, estimate the recovery rate after such perturbations, or assess resilience changes indirectly from stationary time series via indicators of critical slowing down. We present here a sequence of ideal methodological steps in the field of resilience science, and argue how to obtain a consistent and multi-faceted view on ecosystem or climate resilience from Earth observation (EO) data. While EO data offers unique potential to study ecosystem resilience globally at high spatial and temporal scale, we emphasize some important limitations, which are associated with the theoretical assumptions behind diagnostic methods and with the measurement process and pre-processing steps of EO data. The latter class of limitations include gaps in time series, the disparity of scales, and issues arising from aggregating time series from multiple sensors. Based on this assessment, we formulate specific recommendations to the EO community in order to improve the observational basis for ecosystem resilience research.

Unstructured citizen science reduces the perception of butterfly local extinctions: the interplay between species traits and user effort

The detection of local extinctions is often hindered by the lack of long-term monitoring schemes, and thus relies on time series of presence data. Recently, citizen science has repeatedly shown its value in documenting species occurrences. We investigated the effectiveness of unstructured citizen science records in reducing the perception of local extinctions in butterfly populations across Italian National Parks. We addressed three research questions: (i) the ability of citizen science data to supplement existing knowledge to complete time series of occurrences, (ii) the impact on data collection of three species features (species size, distribution and length of flight period) determining their appearance, and (iii) the interplay between participant effort and species appearance in the amount of diversity recorded on the iNaturalist platform. Our analysis of 98,922 records of Italian butterflies (39,929 from literature and 58,993 from iNaturalist of which 7427 from National Parks) showed that the addition of iNaturalist data filled many recent gaps in time series, thus reducing the perception of potential local extinctions. Records from more engaged users encompassed a higher fraction of local biodiversity and were more likely to reduce the perception of local extinctions. User effort strongly interacted with species features in determining the frequency of records for individual species. In particular, more engaged users were less affected by species size. We provided updated butterfly checklists for Italian National Parks and a R package to calculate potential extinction upon time series. These results offer guidance for protected areas, conservationists, policymakers, and citizen scientists to optimize monitoring of local populations.

Enhancing water use efficiency in precision irrigation: data-driven approaches for addressing data gaps in time series

Real-time soil matric potential measurements for determining potato production's water availability are currently used in precision irrigation. It is well known that managing irrigation based on soil matric potential (SMP) helps increase water use efficiency and reduce crop environmental impact. Yet, SMP monitoring presents challenges and sometimes leads to gaps in the collected data. This research sought to address these data gaps in the SMP time series. Using meteorological and field measurements, we developed a filtering and imputation algorithm by implementing three prominent predictive models in the algorithm to estimate missing values. Over 2 months, we gathered hourly SMP values from a field north of the Péribonka River in Lac-Saint-Jean, Québec, Canada. Our study evaluated various data input combinations, including only meteorological data, SMP measurements, or a mix of both. The Extreme Learning Machine (ELM) model proved the most effective among the tested models. It outperformed the k-Nearest Neighbors (kNN) model and the Evolutionary Optimized Inverse Distance Method (gaIDW). The ELM model, with five inputs comprising SMP measurements, achieved a correlation coefficient of 0.992, a root-mean-square error of 0.164 cm, a mean absolute error of 0.122 cm, and a Nash-Sutcliffe efficiency of 0.983. The ELM model requires at least five inputs to achieve the best results in the study context. These can be meteorological inputs like relative humidity, dew temperature, land inputs, or a combination of both. The results were within 5% of the best-performing input combination we identified earlier. To mitigate the computational demands of these models, a quicker baseline model can be used for initial input filtering. With this method, we expect the output from simpler models such as gaIDW and kNN to vary by no more than 20%. Nevertheless, this discrepancy can be efficiently managed by leveraging more sophisticated models.

Comparison of Three Imputation Methods for Groundwater Level Timeseries

This study compares three imputation methods applied to the field observations of hydraulic head in subsurface hydrology. Hydrogeological studies that analyze the timeseries of groundwater elevations often face issues with missing data that may mislead both the interpretation of the relevant processes and the accuracy of the analyses. The imputation methods adopted for this comparative study are relatively simple to be implemented and thus are easily applicable to large datasets. They are: (i) the spline interpolation, (ii) the autoregressive linear model, and (iii) the patched kriging. The average of their results is also analyzed. By artificially generating gaps in timeseries, the results of the various imputation methods are tested. The spline interpolation is shown to be the poorest performing one. The patched kriging method usually proves to be the best option, exploiting the spatial correlations of the groundwater elevations, even though spurious trends due to the the activation of neighboring sensors at times affect their reconstructions. The autoregressive linear model proves to be a reasonable choice; however, it lacks hydrogeological controls. The ensemble average of all methods is a reasonable compromise. Additionally, by interpolating a large dataset of 53 timeseries observing the variabilities of statistical measures, the study finds that the specific choice of the imputation method only marginally affects the overarching statistics.

Remote Sensing of Poplar Phenophase and Leaf Miner Attack in Urban Forests

Remote sensing of phenology is adopted as the practice in greenery monitoring. Now research is turned towards the fusion of data from various sensors to fill in the gap in time series and allow monitoring of pests and disturbances. Poplar species were monitored for the determination of the best approach for detecting phenology and disturbances. With the adjustments that include a choice of indices, wavelengths, and a setup, a multispectral camera may be used to calibrate satellite images. The image processing pipeline included different denoising and interpolation methods. The correlation of the changes in a signal of top and lateral imaging proved that the contribution of the whole canopy is reflected in satellite images. Normalized difference vegetation index (NDVI) and normalized difference red edge index (NDRE) successfully distinguished among phenophases and detected leaf miner presence, unlike enhanced vegetation index (EVI). Changes in the indices were registered before, during, and after the development of the disease. NDRE is the most sensitive as it distinguished among the different intensities of damage caused by pests but it was not able to forecast its occurrence. An efficient and accurate system for detection and monitoring of phenology enables the improvement of the phenological models’ quality and creates the basis for a forecast that allows planning in various disciplines.

Reconstructing global PM2.5 monitoring dataset from OpenAQ using a two-step spatio-temporal model based on SES-IDW and LSTM

Fine particulate matter (PM2.5) is widely concerned for its harmful impacts on global environment and human health, making air pollution monitoring so crucial and indispensable. As the world’s first open, real-time, and historical air quality platform, OpenAQ collects and provides government measurement and research-level data from various channels. However, despite OpenAQ’s innovation in providing us with ground-measured PM2.5 worldwide, we find significant data gaps in time series for most of the sites. The incompleteness of the data directly affects the public perception of PM2.5 concentration levels and hinders the progress of research related to air pollution. To address these issues, a two-step hybrid model named ST-SILM, i.e. spatio-temporal model with single exponential smoothing-inverse distance weighted (SES-IDW) and long short-term memory (LSTM), is proposed to repair the missing data from PM2.5 sites worldwide collected from OpenAQ from 2017 to 2019. Both spatio-temporal correlation and neighborhood fields are considered and established in the model. To be specific, SES-IDW were firstly used to repair missing values, and secondly, the LSTM network was employed to reconstruct the time series of continuous missing data. After the global ground-measured PM2.5 was reconstructed, the light gradient boosting machine model was applied to remote sensing estimation of the original ground-measured PM2.5 and of the reconstructed ground-measured PM2.5 to further verify the performance of ST-SILM. Experiment results show that the estimation accuracy of the reconstructed dataset is better (R 2 from 2017 to 2019 increased by 0.02, 0.02, and 0.01 compared with the original dataset). Therefore, it is concluded that the proposed model can effectively reconstruct data from PM2.5 sites worldwide.

A novel regression method for harmonic analysis of time series

Harmonic analysis of time series is an important technique to reveal seasonal land surface dynamics using remote sensing information. However, frequency selection in the harmonic analysis is often difficult because high-frequency components are useful for delineating seasonal dynamics but sensitive to noise and gaps in time series. On the other hand, it is challenging to obtain temporally continuous satellite data with high quality because of atmospheric contamination. We developed a novel regression method named Harmonic Adaptive Penalty Operator (HAPO) for harmonic analysis of unevenly distributed time series. We introduced a new penalty function to minimize unexpected fluctuations in the model, which can substantially reduce the overfitting issue of regression in time series with temporal gaps. Specifically, the new penalty function minimizes the length of the model curve and the value range difference between the model and time series observations. We compared HAPO with three widely used regression methods (OLS: Ordinary Least Squares; LASSO: Least Absolute Shrinkage and Selection Operator; and Ridge) with different scenarios using Landsat time series data across the United States. First, we evaluated methods using Landsat surface reflectance time series within a single year. HAPO showed small and consistent monthly Root Mean Square Deviation (RMSD) values, in which most of the time RMSD values of predicted reflectance were less than 0.04. More importantly, HAPO showed consistent and less bias given varying density and irregularity of time series. Second, we evaluated methods using multi-year time series and the result suggested that HAPO was a better predictor of relatively short time series (less than4 years) with steady small RMSD values. When a longer time series (≥4 years) was used, all four methods disclosed similar RMSD values, but HAPO outperformed other three methods when there were temporal gaps. Last, we preliminarily tested how regression methods affected change detection and classification accuracy. HAPO showed the highest change detection accuracy of all tests in terms of F1 score when using the change threshold of 0.9999. In classification, HAPO produced the highest accuracy for short time series segments (one- or two-year time series). In contrast, all methods reached similar accuracy for 5-year time series. These results suggest that for areas that have large seasonal observation gaps or for time series that have less than 4 years records, HAPO can provide more consistent and accurate analytical results than other regression methods for harmonic analysis of time series.

Effective method for filling gaps in time series of environmental remote sensing data: An example on evapotranspiration and land surface temperature images

Phenomena such as cloudiness, atmospheric aerosol or sensor failure cause missing data (gaps) in remote sensing images and damage spatial and temporal continuity. Dealing with this deficiency is of importance for continuous spatio-temporal modeling and environmental studies. Simplicity, efficiency and accuracy are dominant factors in practicality of gap filling algorithms especially in dealing with large gaps and long time series. In this study, an effective and efficient spatio-temporal gap filling algorithm is proposed, implemented and tested. The method was applied to MODIS 8-day Land Surface Temperature (LST) and Evapotranspiration (ET) datasets with a 1 km spatial resolution. To assess the performance of the proposed methods, artificial gaps were introduced and filled. Then estimated and real values were compared. The results showed that our method can predict the missing values very accurately (based on RMSE) even in gaps with heterogeneous surface for both variables. Our proposed method has no limitation on the shape and size of the gaps. The proposed algorithm is flexible regarding parameterization. It can handle large volumes of datasets due to its parallelizable structure. More importantly, the method run-time is extremely low even in large gaps.

Spectral Recalibration of NOAA HIRS Longwave CO2 Channels toward a 40+ Year Time Series for Climate Studies

The High-Resolution Infrared Radiation Sounder (HIRS) on NOAA and MetOp A/B satellites has been observing the Earth continuously for over four decades, providing essential data for operational numerical weather prediction, retrieval of atmospheric vertical profile, and total column information on atmospheric temperature, moisture, water vapor, ozone, cloud climatology, and other geophysical parameters globally. Although the HIRS data meets the needs of the short-term weather forecast, there are inconsistencies when the long-term decadal time series is used for time series analysis. The discrepancies are caused by several factors, including spectral response differences between the HIRS models on the satellites and spectral response uncertainties and other calibration issues. Previous studies have demonstrated that significant improvements can be achieved by recalibrating some of the HIRS longwave CO2 channels (Channels 4, 5, 6, and 7), which has helped make the time series more consistent. The current study aims to extend the previous study to the remaining longwave infrared sounding channels, including Channels 1, 2, 3, and 8, using a similar approach. Similar to previous findings, the spectral shift of the HIRS bands has helped improve the consistency in the time series from NOAA-06 to MetOp-A and B for these channels. We also found that HIRS channels on MetOp-B also have bias relative to Infrared Atmospheric Sounding Interferometer (IASI) on the same satellite, especially Channel 4, and a spectral shift significantly reduced the bias. To bridge the observation gap in time series in the mid-1980s between NOAA-07 and NOAA-09, the global mean method has been used since no transfer radiometers between them was available for this period, and the spectral response function corrections, therefore, can be applied to the earliest satellites (NOAA-06) for these channels. The recalibration parameters have been provided to other scientists at the University of Wisconsin for improving the time series in their long-term studies using historical HIRS data and are now made available to the science community.

A Regional Neural Network Approach to Estimate Water-Column Nutrient Concentrations and Carbonate System Variables in the Mediterranean Sea: CANYON-MED

A regional neural network-based method, “CANYON-MED” is developed to estimate nutrients and carbonate system variables specifically in the Mediterranean Sea over the water column from pressure, temperature, salinity, and oxygen together with geolocation and date of sampling. Six neural network ensembles were developed, one for each variable (i.e., three macronutrients: nitrates (NO3-), phosphates (PO43-) and silicates (SiOH4), and three carbonate system variables: pH on the total scale (pHT), total alkalinity (AT), and dissolved inorganic carbon or total carbon (CT), trained using a specific quality-controlled dataset of reference “bottle” data in the Mediterranean Sea. This dataset is representative of the peculiar conditions of this semi-enclosed sea, as opposed to the global ocean. For each variable, the neural networks were trained on 80% of the data chosen randomly and validated using the remaining 20%. CANYON-MED retrieved the variables with good accuracies (Root Mean Squared Error): 0.78 μmol.kg–1 for NO3-, 0.043 μmol.kg–1 for PO43- and 0.71 μmol.kg–1 for Si(OH)4, 0.014 units for pHT, 13 μmol.kg–1 for AT and 12 μmol.kg–1 for CT. A second validation on the ANTARES independent time series confirmed the method’s applicability in the Mediterranean Sea. After comparison to other existing methods to estimate nutrients and carbonate system variables, CANYON-MED stood out as the most robust, using the aforementioned inputs. The application of CANYON-MED on the Mediterranean Sea data from autonomous observing systems (integrated network of Biogeochemical-Argo floats, Eulerian moorings and ocean gliders measuring hydrological properties together with oxygen concentration) could have a wide range of applications. These include data quality control or filling gaps in time series, as well as biogeochemical data assimilation and/or the initialization and validation of regional biogeochemical models still lacking crucial reference data. Matlab and R code are available at https:// github.com/MarineFou/CANYON-MED/.

Filling the gaps of in situ hourly PM<sub>2.5</sub> concentration data with the aid of empirical orthogonal function analysis constrained by diurnal cycles

Abstract. Data gaps in surface air quality measurements significantly impair the data quality and the exploration of these valuable data sources. In this study, a novel yet practical method called diurnal-cycle-constrained empirical orthogonal function (DCCEOF) was developed to fill in data gaps present in data records with evident temporal variability. The hourly PM2.5 concentration data retrieved from the national ambient air quality monitoring network in China were used as a demonstration. The DCCEOF method aims to reconstruct the diurnal cycle of PM2.5 concentration from its discrete neighborhood field in space and time firstly and then predict the missing values by calibrating the reconstructed diurnal cycle to the level of valid PM2.5 concentrations observed at adjacent times. The statistical results indicate a high frequency of data gaps in our retrieved hourly PM2.5 concentration record, with PM2.5 concentration measured on about 40 % of the days suffering from data gaps. Further sensitivity analysis results reveal that data gaps in the hourly PM2.5 concentration record may introduce significant bias to its daily averages, especially during clean episodes at which PM2.5 daily averages are observed to be subject to larger uncertainties compared to the polluted days (even in the presence of the same amount of missingness). The cross-validation results indicate that our suggested DCCEOF method has a good prediction accuracy, particularly in predicting daily peaks and/or minima that cannot be restored by conventional interpolation approaches, thus confirming the effectiveness of the consideration of the local diurnal variation pattern in gap filling. By applying the DCCEOF method to the hourly PM2.5 concentration record measured in China from 2014 to 2019, the data completeness ratio was substantially improved while the frequency of days with gapped PM2.5 records reduced from 42.6 % to 5.7 %. In general, our DCCEOF method provides a practical yet effective approach to handle data gaps in time series of geophysical parameters with significant diurnal variability, and this method is also transferable to other data sets with similar barriers because of its self-consistent capability.

Indirect prediction system for variables that have gaps in their time series

Abstract Gaps in time series as well as the absence of such series make the implementation of prediction system difficult. This paper proposes a new methodology to fill gaps in time series that do not present fixed sampling rate. This paper also proposes the development of two forecast models for time series. The first model is based on autoregressive multilayer neural network that uses only the desired time series, while the second one is developed with multilayer neural network that uses pattern recognition in order to perform indirect predictions of a certain variable. Therefore, the second model does not need the variable time series to make predictions, but any time series that has correlation with the desired variable. The methodology is tested in limnological variables collected in the Paraguay River since 1987, and the results observed in each process are presented in order to validate the methodology of gap filling and forecast used.

Spatio-Temporal Reconstruction of MODIS NDVI by Regional Land Surface Phenology and Harmonic Analysis of Time-Series

Satellite-derived Normalized Difference Vegetation Index (NDVI) is frequently obstructed by adverse atmospheric components resulting in data gaps in time series. The harmonic analysis of time series (HANTS) algorithm is a widely used technique to reconstruct missing NDVI time series. However, due to restriction of HANTS to act within temporal dimension, its direct application is bound to endure practical problems in spatiotemporal reconstruction due to large data gaps. This study proposes Moving Offset Method (MOM), a novel prefilling method applied on NDVI time series prior to application of HANTS. MOM restores the missing NDVI time series by assuming that it tends to follow a reference pattern of land surface phenology (NDVI ref ). The NDVI ref is prepared by using a recursive search and fill algorithm (SFA) for data availability without null values. It restores null values in NDVI ref at a pixel by using coefficients of linear regression with NDVI ref at another pixel having identical conditions. Finally, the prefilling is prior to application of HANTS. The proposed approach is demonstrated by using MODIS 16-daily time series data for Northeast India and Bhutan region which is covered with frequent seasonal clouds. Besides direct application of HANTS, it is also compared with similar approaches which includes prefilling by inverse distance weighted (IDW) and cubic spline, prior to application of HANTS. The fitting indicators, overall reconstruction error (ORE) and normalized noise related error (NNRE) are found to be best for proposed approach in spatiotemporal comparison. Also, restoration of seasonality trait the NDVI time series better in the proposed approach. This approach is concluded to be an enhancement for HANTS that could be helpful in improving quality of NDVI reconstruction for regions with frequent seasonal obstructions around the globe.

DeepAnT: A Deep Learning Approach for Unsupervised Anomaly Detection in Time Series

Traditional distance and density-based anomaly detection techniques are unable to detect periodic and seasonality related point anomalies which occur commonly in streaming data, leaving a big gap in time series anomaly detection in the current era of the IoT. To address this problem, we present a novel deep learning-based anomaly detection approach (DeepAnT) for time series data, which is equally applicable to the non-streaming cases. DeepAnT is capable of detecting a wide range of anomalies, i.e., point anomalies, contextual anomalies, and discords in time series data. In contrast to the anomaly detection methods where anomalies are learned, DeepAnT uses unlabeled data to capture and learn the data distribution that is used to forecast the normal behavior of a time series. DeepAnT consists of two modules: time series predictor and anomaly detector. The time series predictor module uses deep convolutional neural network (CNN) to predict the next time stamp on the defined horizon. This module takes a window of time series (used as a context) and attempts to predict the next time stamp. The predicted value is then passed to the anomaly detector module, which is responsible for tagging the corresponding time stamp as normal or abnormal. DeepAnT can be trained even without removing the anomalies from the given data set. Generally, in deep learning-based approaches, a lot of data are required to train a model. Whereas in DeepAnT, a model can be trained on relatively small data set while achieving good generalization capabilities due to the effective parameter sharing of the CNN. As the anomaly detection in DeepAnT is unsupervised, it does not rely on anomaly labels at the time of model generation. Therefore, this approach can be directly applied to real-life scenarios where it is practically impossible to label a big stream of data coming from heterogeneous sensors comprising of both normal as well as anomalous points. We have performed a detailed evaluation of 15 algorithms on 10 anomaly detection benchmarks, which contain a total of 433 real and synthetic time series. Experiments show that DeepAnT outperforms the state-of-the-art anomaly detection methods in most of the cases, while performing on par with others.

Filling gaps in time series of space-geodetic positioning

Several methods of time series analysis and forecasting require data at regular time intervals. But in space geodesy, most datasets are often full of gaps, resulting for example from hardware issue, modification of models, change of analysis strategy, and local geophysical phenomenon. The purpose of this paper is to fill the gaps in time series of space-geodetic station positions, by the use of two different approaches: the iterative singular spectrum analysis (ISSA) and the generalized regression neural network (GRNN). In order to test the efficiency of the proposed methods to properly process missing data, we created synthetic gaps at random points in regular time series (i.e., time series without gaps) of Global Positioning System (GPS) and Doppler Orbitography and Radiopositioning Integrated by Satellite (DORIS) station positions with data span longer than 4 years. For each analyzed time series, we created gaps (by removing successive points) of different lengths ranging from 1 to 52 gaps, and then, we filled these gaps by ISSA, GRNN, and other classical methods of interpolation such as nearest neighbor, linear, and cubic interpolations. The interpolation precision was evaluated by the technique of cross-validation which compares the estimated values with the original data. After several simulations on position time series with different lengths, we found that the ISSA technique provides better results in terms of root mean square error.

Interpolation in Time Series: An Introductive Overview of Existing Methods, Their Performance Criteria and Uncertainty Assessment

A thorough review has been performed on interpolation methods to fill gaps in time-series, efficiency criteria, and uncertainty quantifications. On one hand, there are numerous available methods: interpolation, regression, autoregressive, machine learning methods, etc. On the other hand, there are many methods and criteria to estimate efficiencies of these methods, but uncertainties on the interpolated values are rarely calculated. Furthermore, while they are estimated according to standard methods, the prediction uncertainty is not taken into account: a discussion is thus presented on the uncertainty estimation of interpolated/extrapolated data. Finally, some suggestions for further research and a new method are proposed.

Kriging-based approach to predict missing air temperature data

The geo-statistical Kriging method is conventionally used in the spatial dimension to predict missing values in a series by utilizing information from neighbouring data, supported by the hypothesis that mathematical expectation is a function of distance between observations. By using a data-driven machine learning-based inferencing and exploration framework, this research applies a Kriging-based interpolation in the temporal dimension to fill in data gaps in time-series of air temperatures. It assesses its performance for artificial gap scenarios (ranging in length from single one to six consecutive data points) generated using data with both daily and hourly resolutions from five sites in Europe (Laqueuille, France; Grillenburg, Germany; Monte Bondone, Italy; Oensingen, Switzerland; Rothamsted, United Kingdom) and one in France overseas (Sedael, Réunion Island). Results show that the method is capable of predicting missing temperatures with acceptable accuracy, especially with the hourly resolution and for non-high elevation sites: modeling efficiency (EF≤1, optimum) >0.8, with the exception of Monte Bondone, placed at >2000ma.s.l. (EF<0). With daily data, maximum temperature was correctly predicted at all sites (0.6≤EF<0.9), while some less accuracy (down to EF<0.4) was noted when predicting missing daily minimum temperatures. In conclusion, the method appears suitable to be applied to fill in hourly temperature gaps, requiring more stringent hypotheses concerning daily data and mountain sites (but further studies are required to draw concluding recommendations).

Sampling strategies for high quality time-series of climatic variables in forest resource assessment

Abstract: Many ecological studies require long-term time series of high quality. Missing data may represent a serious problem since they can affect the reliability of measured variables in specific locations. To which extent and according to which methodology a gap in time series should be filled is a major research challenge. In this study, the time-series of meteorological data relative to 13 monitoring sites from the ICP-Forest network in Italy were analysed with the aim to define the minimum number of site-specific observations, which can be considered adequate for further analysis on forest resource management. Three main climatic variables were taken into account in the analysis: air temperature, relative humidity and total precipitation. By using an increasing proportion of available data, descriptive and inferential statistic methods were applied to evaluate the amount of variability along the period of analysis (1998-2013) and associated error of estimation at seasonal level. The relative importance of each factor accounted in our analysis (season, year, variable, plot, sampling proportion) was investigated fitting a Random Forest model on the results of the bootstrapping procedure. Air temperature was the variable with a marked seasonal profile and the easiest to be represented at monthly level on a specific time period. Humidity and precipitation were more stable across the analysed time period. Trends in precipitation showed that a high amount of variability could be detected only when > 80% of valid observations were available. Humidity showed an intermediate pattern, with an exponential increase in the amount of explained variability when using an increased proportion of sampled observations. Random Forest Regression models indicated sampling proportion (i.e., number of available observations) as an important factor for trend analysis of relative air humidity and precipitation. We conclude that monthly or seasonal statistics can be proficiently estimated for both air temperature and relative humidity with a proportion of missing values higher than 50%. Conversely, a reliable analysis of intra-seasonal or intra-monthly precipitation variability requires a much higher amount of observations. In the latter case gap filling represents the only feasible solution.

Empirical Sewer Water Quality Model for Generating Influent Data for WWTP Modelling

Wastewater treatment plants (WWTP) typically have a service life of several decades. During this service life, external factors, such as changes in the effluent standards or the loading of the WWTP may change, requiring WWTP performance to be optimized. WWTP modelling is widely accepted as a means to assess and optimize WWTP performance. One of the challenges for WWTP modelling remains the prediction of water quality at the inlet of a WWTP. Recent applications of water quality sensors have resulted in long time series of WWTP influent quality, containing valuable information on the response of influent quality to e.g., storm events. This allows the development of empirical models to predict influent quality. This paper proposes a new approach for water quality modelling, which uses the measured hydraulic dynamics of the WWTP influent to derive the influent water quality. The model can also be based on simulated influent hydraulics as input. Possible applications of the model are filling gaps in time series used as input for WWTP models or to assess the impact of measures such as real time control (RTC) on the performance of wastewater systems.

Quality control and data restoration of metocean Arctic data

We analyze the problem of data availability and how this situation (which is very common in practice) is a limiting factor for doing quality control and data restoration of MetOcean fields. In order to support our claim, we analyzed the case of data availability in the Arctic Region, since it is true that climate historical archives have always been scarce in this area, due to the harsh climate conditions and logistic difficulties that involve the deployment of in-situ weather systems in the region. In addition to the problem of data scarcity, we also have the problem of inconsistent and missing observations. In this paper, we highlight this problem by the analysis of 125 hourly temperature and hourly wind speed time series, which have been all in common their spatial distribution and present an evaluation of those data in terms of quality and availability. We also discuss the way the presence of gaps in time series can condition the ability of estimating synthetic data and discuss possible solutions to overcome this problem in areas where no neighboring stations are available.