Environmental Time Series Data Research Articles

A template framework for environmental timeseries data acquisition

Environmental timeseries data variety is exploding in the Internet of Things era, making data reuse a very demanding task. Data acquisition and integration remains a laborious step of the environmental data lifecycle. Environmental data heterogeneity is a persistent issue, as data are becoming available through different protocols and stored under diverse, custom formats. In this work, we deal with syntactic heterogeneity in environmental timeseries data. Our approach is based on describing different dataset syntaxes using abstract representations, called templates. We designed and implemented EDAM (Environmental Data Acquisition Module), a template framework that facilitates timeseries data acquisition and integration. EDAM templates are written using programming language-agnostic semantics, and can be reused both for input and output, thus enabling data reuse via transformations across different formats. We demonstrate EDAM generality in seven case studies, which involve scraping online data, extracting observations from a relational database, or aggregating historical timeseries stored in local files. Case studies span different environmental sciences domains, including meteorology, agriculture, urban air quality and hydrology. We also demonstrate EDAM for data dissemination, as instructed by output templates. We identified several syntactic interoperability challenges though the case studies, that include managing with differences in formatting observables, temporal and spatial references, and metadata documentation, and addressed them with EDAM. EDAM implementation has been released under an open-source license.

The recovery of estuarine quality and the perceived increase of cultural ecosystem services by beach users: A case study from northern Spain

In Europe, the quality of coastal bathing waters improved considerably in the last decades, mainly due to the more demanding legislation and the adoption of water sanitation plans. In the Nerbioi estuary (North Spain), the Wastewater Treatment Plan implemented between 1990 and 2001 resulted on an abrupt decrease in microbial concentration; thus, complying with bathing waters legislation and allowing recreational activities again in the three beaches of the estuary. However, little is known about how improvements in bathing waters influences the provision of cultural ecosystem services and human well-being. A questionnaire was used to study beach users' behaviour and perceptions and compared with environmental time-series data (microbial concentration and water transparency). Most respondents perceived an improvement in bathing waters quality and linked it to the estuarine sanitation. Nerbioi beaches are important recreational areas, mainly for local visitors, and water quality improvement was found to be a critical factor for deciding to visit these beaches. Furthermore, most visitors answered that they would not return if water conditions deteriorate. Significant differences existed between beaches, with the most inner beach presenting worse environmental conditions than the other two beaches; and matching user's perceptions. Our findings highlight that water sanitation actions are important for the recovery of degraded coastal environments and for the maintenance of ecosystem services. Also, that multidisciplinary research is necessary to better comprehend the links between environmental recovery and the provision of ecosystem services.

Generalizable, Extensible Methods to Implement QA/QC Tests on Environmental Data

Abstract. As environmental data are increasingly recognized as important, increased attention is focused on data quality. A critical component of quality assurance and quality control (QA/QC) is implementing tests for problems like missing data, outliers, and changes that might be too abrupt or too slow to be real, and most implementations involve coding the combinational logic of tests in if-then-else constructs to arrive at outcomes or actions depending on the inputs. Herein we propose two alternative implementation methods that avoid if-then-else logic and therefore can be used to perform more complex QA/QC screens on data from stations with one or more sensors. The objectives of this technical note are to describe the methods and to demonstrate their performance using realistic examples with existing data. Both new methods use truth tables to express and implement logic; the first method expands a truth table with N tests to all 2N possible combinations and uses that as a lookup table, while the second method matches test patterns in the truth table using its original unexpanded form. The if-then-else method and the two new methods are illustrated with a simple example of air temperature measurement for scenarios with one, two, or three (redundant) sensors. The relative advantages and disadvantages of each method are discussed. Scalability of the new methods is illustrated using measurements from a local weather station. If provided both the test results and either the expanded truth table (for the lookup method) or the original truth table (for the pattern-match method), the core code was general. The flexibility of these methods should increase the power of QA/QC programs for environmental or other time series data. Keywords: Multiple sensors, QA/QC, Quality assurance, Quality control, Truth table, Weather data, Weather station.

Confined Ecological Predictors of Cholera in Bangladesh and Vietnam

Environmental parameters are the main factors for the development of cholera outbreak because it has relation based on local environment, so it is necessary to prove and improve model for predictors for its prevention. The distribution of cholera is based on environmental predators, in Bangladesh and Vietnam, cholera outbreak relationships are examined by satellite-derived and in-situ environmental time series data ordered probit and probit models were used to examine the cholera associations in Bangladesh and at 2 sites in Vietnam respectively. With the existing data of Bangladesh shows, increase in chlorophyll concentration along with increase in cholera magnitude. Whereas, in Vietnam SST increases in Hue with increase in river height at Nha Trang regions. Cholera appearance and epidemic magnitude are related to the local environment parameters, but in certain situations cholera epidemics are rare at differential environmental effects.

Modelling Lagged Associations in Environmental Time Series Data: A Simulation Study.

This study assesses two alternative approaches for investigating linear and nonlinear lagged associations in environmental time series data, comparing through simulations simple methods based on moving average summaries with more flexible distributed lag linear and nonlinear models. Results indicate that the latter provide estimates with no or low bias and close-to-nominal confidence intervals, even for long-lagged associations and in the presence of strong seasonal trends. Moving average models represent a viable alternative only in the presence of relatively short lag periods, and when the lag interval is correctly specified. In contrast, the use of moving averages to roughly approximate long and complex lag patterns, or the specification of an interval different than the actual lag period, can result in substantial biases. More flexible approaches based on distributed lag linear or nonlinear models provide noteworthy advantages, in particular when complex lagged associations are assumed.

A simulation study on methods for modelling lagged associations in environmental time series data

Introduction Alternative methods have been proposed for modelling lagged associations in time series data, from simple moving average summaries to distributed lag functions. However, little is known about their absolute and comparative performance. In particular, doubts have been raised on the validity of estimates suggesting strong and long-lagged associations with cold temperature, due to supposed confounding effects of seasonality. Methods In this simulation study, methods based on moving average summaries are compared with more flexible distributed lag linear (DLM) and non-linear (DLNM) models. The analysis is based on range of scenarios of linear or non-linear associations with air pollution (ozone) or temperature, respectively, each characterized by short or long lags. For each scenario, 5000 replicates of series of daily mortality counts were simulated and then analysed in conventional time series models. The two methods were assessed and compared in terms of bias, coverage and precision. Results Results indicate that DLM and DLNM provide estimates with no or low bias and close-to-nominal confidence intervals, even for long-lagged associations and in the presence of strong seasonal trends. Moving average models represent a viable alternative only in the presence of relatively short lag periods, and when the lag interval is correctly specified. In contrast, the use of moving averages to roughly approximate long and complex lag patterns, or the specification of an interval different than the actual lag period, can result in substantial biases. Conclusions Flexible approaches based on distributed lag linear or non-linear models are valuable tools for estimating associations between environmental factors and health, and provide significant advantages compared to simpler approaches, in particular when complex lagged associations are assumed. These findings also reassure about the validity of recent estimates of historical and projected health impacts of cold temperature.

Spatiotemporal variation in flow-dependent recruitment of long-lived riverine fish: Model development and evaluation

Natural flow regimes can play a major role as an overarching ecosystem driver in reproduction and recruitment of riverine fishes. Human needs for freshwater however have altered hydrology of many riverine systems worldwide, threatening fish population sustainability. To understand and predict how spatiotemporal dynamics of flow regimes influence reproductive and recruitment variability, and ultimately population sustainability of shovelnose sturgeon (Scaphirhynchus platorynchus), we develop a spatially explicit (1D) individual-based population model that mechanistically (via energetics-based processes) simulates daily activities (dispersal, spawning, foraging, growth, and survival). With field observations of sturgeon and habitat conditions in a major tributary of the Missouri River system (USA), we calibrate and evaluate the model via pattern-oriented modeling. Model simulation experiments using 17-year environmental time series data showed that seasonal and interannual variation in hydrological conditions plays a major role in timing, location, and magnitude of spawning and recruitment success of sturgeon. During droughts, consecutive weak year-classes resulted in a steady population decline. While low flow and subsequent low prey production limited foraging opportunities and slowed gonad development, these conditions were not severe enough for adults to abort the reproductive cycle. Post-settlement larval sturgeon were however unable to feed efficiently to grow out of a size-dependent ‘predation window’, resulting in high mortality. Slow growth and low survival of larval sturgeon thus likely play a larger role in recruitment failures during droughts than low or lack of spawning events.

An analytical procedure for multi-site, multi-season streamflow generation using maximum entropy bootstrapping

Stochastic time series models are very useful in many environmental domains. In this paper, an analytical procedure for multi-site, multi-season streamflow generation using maximum entropy bootstrap stochastic model (M3EB) is developed that can implicitly preserve both the spatial and temporal dependence structure, in addition to the other statistical characteristics present in the historical time series. The proposed model is computationally less demanding and simple in terms of modeling complexity. The maximum entropy bootstrap (MEB) generates random samples from the empirical cumulative distribution function (ECDF) and rearranges the random series based on the rank ordering of the historical time series. The modeling structure of MEB implicitly satisfies the ergodic theorem (preservation of summary statistics) and guarantees the reproduction of the time dependent structure of an underlying process. The orthogonal transformation is used with M3EB to capture the spatial dependence present in the multi-site collinear data. The performance of M3EB is verified by comparing the statistical characteristics between the observed and synthetically generated streamflows. Three case studies from Colorado River Basin, USA; Red River Basin, USA and Canada; and Cauvery River Basin, India; are used to demonstrate the advantages of M3EB. The statistical measures adopted for evaluation of M3EB performance include monthly statistics (mean, standard deviation and skewness), temporal and spatial correlation, smoothing (flows other than present in historical data) and extrapolation (flows outside the range of historical data). The M3EB model shows (i) a high level of accuracy in preserving the statistics; and (ii) a high computational efficiency. Since M3EB can be used for multiple variable problems, the model can be easily extended to other environmental or hydroclimatic time series data.

MoBILAB: an open source toolbox for analysis and visualization of mobile brain/body imaging data

A new paradigm for human brain imaging, mobile brain/body imaging (MoBI), involves synchronous collection of human brain activity (via electroencephalography, EEG) and behavior (via body motion capture, eye tracking, etc.), plus environmental events (scene and event recording) to study joint brain/body dynamics supporting natural human cognition supporting performance of naturally motivated human actions and interactions in 3-D environments (Makeig et al., 2009). Processing complex, concurrent, multi-modal, multi-rate data streams requires a signal-processing environment quite different from one designed to process single-modality time series data. Here we describe MoBILAB (more details available at sccn.ucsd.edu/wiki/MoBILAB), an open source, cross platform toolbox running on MATLAB (The Mathworks, Inc.) that supports analysis and visualization of any mixture of synchronously recorded brain, behavioral, and environmental time series plus time-marked event stream data. MoBILAB can serve as a pre-processing environment for adding behavioral and other event markers to EEG data for further processing, and/or as a development platform for expanded analysis of simultaneously recorded data streams.

Advanced information criterion for environmental data quality assurance

Abstract. A new method for testing time series of environmental data for internal inconsistencies is presented. The method divides the dataset into several disjunct blocks. By means of a comparison of the blocks' estimated probability density distributions, each block is compared with the others. In order to judge the differences, four different measures are used and compared: Kullback-Leibler Divergence, Jensen-Shannon Divergence, Earth Mover's Distance and the Root Mean Square. By looking at the resulting patterns, conclusions on possible inconsistencies in the data can be drawn. This paper shows some sensitivitiy tests and gives an example for an application to real data. Furthermore, it is shown, in which cases of errors (shift in mean, shift in variance and rounding), which measure performs best.

Nonparametric harmonic regression for estuarine water quality data

AbstractPeriodicity is omnipresent in environmental time series data. For modeling estuarine water quality variables, harmonic regression analysis has long been the standard for dealing with periodicity. Generalized additive models (GAMs) allow more flexibility in the response function. They permit parametric, semiparametric, and nonparametric regression functions of the predictor variables. We compare harmonic regression, GAMs with cubic regression splines, and GAMs with cyclic regression splines in simulations and using water quality data collected from the National Estuarine Reasearch Reserve System (NERRS). While the classical harmonic regression model works well for clean, near‐sinusoidal data, the GAMs are competitive and are very promising for more complex data. The generalized additive models are also more adaptive and require less‐intervention. Copyright © 2009 John Wiley & Sons, Ltd.

Local Environmental Predictors of Cholera in Bangladesh and Vietnam

Environmental factors have been shown to be related to cholera and thus might prove useful for prediction. In Bangladesh and Vietnam, temporal cholera distributions are related to satellite-derived and in-situ environmental time series data in order to examine the relationships between cholera and the local environment. Ordered probit models examine associations in Bangladesh; probit models examine associations at 2 sites in Vietnam. Increases in ocean chlorophyll concentration are related to an increased magnitude of cholera in Bangladesh. Increases in sea surface temperature are most influential in Hue, Vietnam, whereas increases in river height have a significant role in Nha Trang, Vietnam. Cholera appearance and epidemic magnitude are related to the local environment. Local environmental parameters have consistent effects when cholera is regular and more prevalent in endemic settings, but in situations where cholera epidemics are rare there are differential environmental effects.

Investigation into the use of the CUSUM technique in identifying changes in mean air pollution levels following introduction of a traffic management scheme

There is an increasing need for statistical techniques to identify and quantify the effects of traffic management schemes on ambient pollution levels. Cumulative sum (CUSUM) charts have been used extensively in industrial process control to detect deviations in production parameters from pre-determined values. This study investigates the use of the CUSUM procedure to identify change in ambient air pollution levels following the introduction of a traffic management scheme at a specific location in Central London. The CUSUM methods of Lucas first compute the standardised deviations of time series observations from the desired process mean. These are accumulated over time to compute the CUSUM at each time point. Data for the analysis were taken from a kerbside monitoring site on Marylebone Road, a six lane trunk route in Central London. In August 2001 the lane adjacent to the monitoring site was designated as a permanent bus lane. The CUSUM analysis clearly identifies a sustained decrease in carbon monoxide concentrations beginning in 2002. However, seasonality and other factors precluded precise characterisation of the timing of the change. When the analysis was repeated using a reference mean that extrapolated the pre-intervention trend in carbon monoxide concentrations, the CUSUM chart no longer identified a sustained decrease. CUSUM appears to offer a simple and rapid method for identifying sustained changes in pollution levels, but the range of confounding influences on carbon monoxide concentrations, most notably underlying trends, seasonality and independent interventions, complicate its interpretation. Its application in assessing the presence or timing of a stepped change in pollution or similar environmental time series data is recommended in its basic form only where the predicted change is large by comparison with other independent influences. The authors believe that further development of the technique beyond this initial study is worthwhile in order to improve the technique's sensitivity.

On case‐crossover methods for environmental time series data

AbstractCase‐crossover methods are widely used for analysing data on the association between health events and environmental exposures. In recent years, several approaches to choosing referent periods have been suggested, with much discussion of two types of bias: bias due to temporal trends, and overlap bias. In the present paper, we revisit the case‐crossover method, focusing on its origin in the case‐control paradigm, in order to throw new light on these biases. We emphasise the distinction between methods based on case‐control logic (such as the symmetric bi‐directional (SBI) method), for which overlap bias is a consequence of non‐exchangeability of the exposure series, and methods based on cohort logic (such as the time‐stratified (TS) method), for which overlap bias does not arise. We show by example that the TS method may suffer severe bias from residual seasonality. This method can be extended to control for seasonality. However, time series regression is more flexible than case‐crossover methods for the analysis of data on shared environmental exposures. We conclude that time series regression ought to be adopted as the method of choice in such applications. Copyright © 2006 John Wiley & Sons, Ltd.

Long-term changes in the fish community structure from the Tsushima warm current region of the Japan/East Sea with an emphasis on the impacts of fishing and climate regime shift over the last four decades

Japanese fisheries production in the Japan/East Sea between 1958 and 2003 increased to their peak (1.76 million tons) in the late 1980s and decreased abruptly with the collapse of Japanese sardine. Catch results for 58 fisheries and various environmental time-series data sets and community indices, including mean trophic level (MTL) and Simpson’s diversity index (DI), were used to investigate the impacts of fishing and climate changes on the structure of the fish community in the Tsushima warm current (TWC) region of the Japan/East Sea. The long-term trend in fisheries production was largely dependent on the Japanese sardine that, as a single species, contributed up to 60% of the total production in the Japanese waters of the Japan/East Sea during the late 1980s. Excluding Japanese sardine, production of the small pelagic species was higher during 1960s and 1990s but lower during 1970s and 1980s. This variation pattern generally corresponds with the trend in water temperature, warmer before early 1960s and after 1990s but colder during 1970s and 1980s. The warm-water, large predatory fishes and cold water demersal species show opposite responses to the water temperature in the TWC region, indicating the significant impact of oceanic conditions on fisheries production of the Japan/East Sea. Declines in demersal fishes and invertebrates during 1970s and 1980s suggested some impact of fishing. MTL and DI show a similar variation pattern: higher during 1960s and 1990s but lower during 1970s and 1980s. In particular, the sharp decline during the 1980s resulted from the abundant sardine catches, suggesting that dominant species have a large effect on the structure of the fish community in the Japan/East Sea. Principal component analysis for 58 time-series data sets of fisheries catches suggested that the fish community varied on inter-annual to inter-decadal scales; the abrupt changes that occurred in the mid-1970s and late 1980s seemed to correspond closely with the climatic regime shifts in the North Pacific. These results strongly suggest that the structure of the fish community in the Japan/East Sea was largely affected by climatic and oceanic regime shifts rather than by fishing. There is no evidence showing “fishing down food webs” in the Japan/East Sea. However, in addition to the impacts of abrupt shifts that occurred in the late 1980s, the large predatory and demersal fishes seem to be facing stronger fishing pressure with the collapse of the Japanese sardine.

Study on Assessing Economic Vulnerability of Small Island Regions

The main purpose of this study is to assess economic vulnerability of small island development regions as part of their sustainability constraints. By combining economic and environmental time series data, we assessed a composite index of economic vulnerability which is constructed from three exogenous variables, namely economic exposure, economic remoteness, and economic impact of environmental and natural disasters. We used the Amami Islands, Kagoshima Prefecture, Japan as the case studies for this paper. The results indicated that using a gross island products based valuation index, Kikaijima is the most vulnerable island in the Amami Islands with a composite economic vulnerability index (CEVI) value of 0.678, while by using a per capita based index, Okinoerabujima is considered the most vulnerable island with a CEVI value of 0.680. From the results we also revealed that smaller islands have relative higher vulnerability than the bigger one, which also confirms some previous country-level vulnerability studies. However, it is matter of fact that some islands that have relatively high vulnerability also have good economic performance as shown by their per capita income. In this regard, it can be argued that the success of these small islands could have been achieved in spite of and not because of their inherent vulnerability conditions as an indicator of sustainability constraint. Regarding these findings, we also examined a comparison between vulnerability results and the preliminary concept of an island's resilience in order to capture another perspective on sustainability assessment in a small island region.

Extinction risk in periodically fluctuating environments

Periodically fluctuating environments occur in various ways in nature but have not, however, been studied in detail yet in the context of the color of environmental noise and extinction risk of populations. We use a stochastic model to simulate population dynamics with compensatory density regulation under four different patterns of periodically fluctuating environments. We found that extinction risk changes dramatically from what was known if the underlying environmental stochasticity driving population dynamics is periodically correlated rather than randomly correlated. Fluctuating environments with a very short period are found to decrease extinction risk over “white noise” fluctuations because a species is never in a bad environment for too long. Conversely, long periods increase extinction risk because species accumulate too much time in a bad environment. Moreover, we found the mean, variance, frequency distribution and especially the extensively studied noise color not to be sufficient for predicting extinction risk in periodically fluctuating environments. Rather, additional attributes of environmental noise have to be considered. The occurrence of monotonic trends within time series of environmental data (e.g. after ‘disturbance’ events), in combination with density regulation, may also affect extinction risk. Our study exemplifies that the investigation of periodically fluctuating environments leads to new insights into the interaction between environmental variation, population dynamics and the resulting extinction risk.

Performance of partial Mann–Kendall tests for trend detection in the presence of covariates

AbstractTrend analyses of time series of environmental data are often carried out to assess the human impact on the environment under the influence of natural fluctuations in temperature, precipitation, and other factors that may affect the studied response variable. We examine the performance of partial Mann–Kendall (PMK) tests, i.e. trend tests in which the critical region is determined by the conditional distribution of one Mann‐Kendall (MK) statistic for monotone trend, given a set of other MK statistics. In particular, we examine the impact of incorporating information regarding covariates in the Hirsch–Slack test for trends in serially correlated data collected over several seasons. Monte Carlo simulation of the performance of PMK tests demonstrates that the gain in power due to incorporation of relevant covariates can be large compared to the loss in power caused by irrelevant covariates. Furthermore, we have found that the asymptotic normality of the test statistics in such tests enables rapid and reliable determination of critical regions, unless the sample size is very small (n < 10) or the different MK statistics are very strongly correlated. A case study of water quality trends shows that PMK tests can detect and correct for rather complex relationships between river water quality and water discharge. The generic character of the PMK tests makes them particularly useful for scanning large sets of data for temporal trends. Copyright © 2002 John Wiley & Sons, Ltd.

Selection bias and confounding in case-crossover analyses of environmental time-series data.

The case-crossover study design is a popular analytic tool for estimating the effects of triggers of acute outcomes by environmental exposures. Although this approach controls for time-invariant confounders by design, it may allow for selection bias and confounding by time-varying factors. We conducted a simulation study of the sensitivity of the symmetric bidirectional case-crossover design to time-varying patterns in exposure and outcome. We identified the effects of selection bias and confounding on symmetric bidirectional case-crossover results and offer strategies to eliminate or reduce these biases. Selection bias results when exposure in the reference periods is not identically representative of exposure in the hazard periods, even when the distribution of exposure is stationary. This bias can be estimated and removed. Selection bias also occurs when the distribution of exposure is nonstationary, but the adjusted symmetric bidirectional case-crossover methodology substantially controls for this. Confounding results from a common temporal pattern in the exposure and the outcome time series, but can also be the result of patterns in exposure and outcome that, although asymptotically uncorrelated, are correlated at finite series lengths. All three biases are reduced by choosing shorter referent-spacing lengths. This effect is illustrated using data on air pollution and daily deaths in Chicago.

Studies on temporal variations of radon in Swedish single-family houses

Two different studies on the temporal variations of radon in Swedish single-family houses overlap with the goal of creating a procedure for using temporal variations of indoor radon and other parameters measured over a short time period to predict longertime averaged indoor radon. The first study has collected detailed environmental and house time series data in a single house over a 3-y time period. This data provide an ideal base for testing the feasibility of using short-time measurements for longer time predictions. The results of the procedure presented here show agreement to within 10% between the actual measured indoor radon over a long time period and the predicted concentrations from shorter time measurements. The second study comprises radon measurements in 158 single-family houses. The data from this study, which are used here, include 11 consecutive one-month integrations of the radon gas concentration in 1993-94, and house ventilation system data. The results demonstrate that the average monthly radon level varies over the year with a minimum in late spring/early summer and a maximum in late autumn. These collective data are inversely dependent on outdoor temperature, except during winter. The same annual pattern is seen regardless of the ventilation system.