Raw Time Series Data Research Articles

A Time Series Sodar Simulator Based on Large-Eddy Simulation

Abstract A sodar simulator capable of producing time series data emulating sodar signals has been developed and tested. The atmospheric fields used to populate the sodar simulator are taken from output of a large-eddy simulation code. The characteristics of the sodar (number and zenith angle of beams, beamwidth, transmit frequency, range resolution, etc.) are defined by the user to allow emulation of existing systems. The range of the reflected acoustic signal is calculated based upon a temperature-dependent speed of sound. Realistic acoustic background noise is simulated using filtered white noise. The raw acoustic time series data are processed using a Fourier transform to yield acoustic Doppler spectra, from which the radial velocities are calculated. The design of the simulator allows for the testing of and comparisons between various signal-processing techniques and averaging periods. An example case of feeding the sodar simulator with large-eddy simulation data representative of a developing convective boundary layer is presented and discussed.

Identification of Quasi-Periodic Nonlinear Waveforms in Turbulent Plasmas

A new method is presented for identifying waveforms of fluctuations in turbulent plasmas. The method is based on heartbeat analysis in which the convolution of a waveform is obtained by employing the phase tracking method. Phase tracking is performed by correlating raw time-series data with a template waveform; the template is evaluated through iteration procedure. The method is applied to fluctuations in a PANTA plasma, and the nonlinear waveform and its distribution of periods are obtained. c

Completeness of the fossil record and the validity of sampling proxies: a case study from the Triassic of England and Wales

Many studies have highlighted correlations between palaeodiversity and sampling proxies. These correlations have been interpreted as evidence for bias, common cause, or redundancy between signals. Here, we compare a number of sampling proxies representing sedimentary rock volume, rock accessibility and worker effort with palaeodiversity through the predominantly terrestrial Triassic System in England and Wales. We find that proxies for sedimentary rock volume and accessibility do not correlate with palaeodiversity until the removal of facies-related preservational and palaeoecological factors. This indicates that a weak sampling signal may be present, but the effects of changing palaeoenvironments are far more important at the regional scale. Significant correlations between worker effort and palaeodiversity are detected, although this is likely to be a result of the preferential sampling of formations already known to be rich in fossils. The fact that there is little evidence for sedimentary rock bias in the fossil record of the Triassic of England and Wales suggests that either (1) sampling bias is not a major source of error at the regional scale or (2) sampling proxies are inadequate representations of geological and human sampling bias. Supplementary material: Correlation test results and raw time series data from the study are available at www.geolsoc.org.uk/SUP18567 .

In-Network Time-Series Data Compression for Electric Internet of Things

The IoT is considered as one of the most important supporting technologies of the smart grid and the smart grid is considered as one of the most important application areas of the IoT. However, the electric IoT has vast amount and many kinds of sensors, very high data acquisition frequency, and a highly heterogeneous network, which lead to the challenge that if the raw time-series data gathered by sensors is all transmitted to the sensing data center via network and then compressed and reserved, the bandwidth and the computing resource requirements of the network and the sensing data center, respectively, are unacceptable. In this paper, in order to cope with the challenge, we propose an in-network time-series data compression algorithm, i.e., the DSDT (Distributed Swinging Door Trending) algorithm, for the electric IoT, which utilizes the computing resource of the sensors to compress the raw sensing data, and then transmits the compressed data to sensing data center where the data will be further compressed and reserved. In this way, the bandwidth and the computing resource requirements of the network and the sensing data center, respectively, are significantly reduced. A performance study shows the superiority of the algorithm.

Data Products at the IRIS DMC: Stepping Stones for Research and Other Applications

In 2010, the Incorporated Research Institutions for Seismology (IRIS) Data Management Center (DMC) expanded the preexisting effort to generate, archive, and distribute data products derived from the extensive data archives of the center, which include the IRIS Program for Array Seismic Studies of the Continental Lithosphere (PASSCAL), Global Seismographic Network (GSN), EarthScope, and many other data sources. This expansion included dedicating two new full‐time staff to data products and to the development of a new system to manage and distribute them. With community guidance provided by a data products working group (DPWG), DMC has developed a number of freely available unique and increasingly popular data products for the seismological community. In addition, DMC is also archiving and distributing other data products that are produced by community members. A complete list of data products managed by DMC is maintained here: http://www.iris.edu/dms/products. Many of these are available from the Searchable Product Depository (SPUD, http://www.iris.edu/spud), the DMC data product management system. Traditionally, data products result from specialized processing applied during research by the principal investigators and their teams. Products span the range of simple plots or animations derived from waveform data to highly refined models. Some waveform‐processing techniques (e.g., receiver functions) have gained wide community use and can be derived by widely accepted algorithms. Such first‐order data products make ideal candidates for routine generation to provide a standard for the user community. Data products are typically derived from raw data to serve as insightful summaries of the raw time series data, or as quality control metrics, or as foundations or stepping stones to more specialized products and research. The overall goal of the DMC data product effort is to produce and manage a variety of community‐approved and utilized data products that complement the raw data managed at the DMC and ultimately serve …

A river system modelling platform for Murray-Darling Basin, Australia

Global climate change and local development make water supply one of the most vulnerable sectors in Australia. The Australian government has therefore commissioned a series of projects to evaluate water availability and the sustainable use of water resources in Australia. This paper discusses a river system modelling platform that has been used in some of these nationally significant projects. The platform consists of three components: provenance, modelling engine and reporting database. The core component is the modelling engine, an agent-based hydrological simulation system called the Integrated River System Modelling Framework (IRSMF). All configuration information and inputs to IRSMF are recorded in the provenance component so that modelling processes can be reproduced and results audited. The reporting database is used to store key statistics and raw output time series data for selected key parameters. This river system modelling platform has for the first time modelled a river system at the basin level in Australia. It provides practitioners with a unique understanding of the characteristics and emergent behaviours of river systems at the basin level. Although the platform is purpose-built for the Murray-Darling Basin, it would be easy to apply it to other basins by using different river models to model agent behaviours.

Influences of various calculation options on heat, water and carbon fluxes determined by open- and closed-path eddy covariance methods

ABSTRACTSynthesis studies using multiple-site datasets for eddy covariance potentially contain uncertainties originating from the use of different flux calculation options, because the choice of the process for calculating half-hourly fluxes from raw time series data is left to individual researchers. In this study, we quantified the uncertainties associated with different flux calculation methods at seven sites. The differences in the half-hourly fluxes were small, generally of the order less than a few percentiles, but they were substantial for the annual fluxes. After the standardisation under current recommendations in the FLUXNET communities, we estimated the uncertainties in the annual fluxes associated with the flux calculations to be 2.6±2.7 W m−2 (the mean 90% ± confidence interval) for the sensible heat flux, 72±37 g C m−2 yr−1 for net ecosystem exchange (NEE), 12±6% for evapotranspiration, 12±6% for gross primary productivity and 16±10% for ecosystem respiration. The self-heating correction strongly influenced the annual carbon balance (143±93 g C m−2 yr−1), not only for cold sites but also for warm sites, but did not fully account for differences between the open- and closed-path systems (413±189 g C m−2 yr−1).

Correlation of pre-earthquake electromagnetic signals with laboratory and field rock experiments

Abstract. Analysis of the 2007 M5.4 Alum Rock earthquake near San José California showed that magnetic pulsations were present in large numbers and with significant amplitudes during the 2 week period leading up the event. These pulsations were 1–30 s in duration, had unusual polarities (many with only positive or only negative polarities versus both polarities), and were different than other pulsations observed over 2 years of data in that the pulse sequence was sustained over a 2 week period prior to the quake, and then disappeared shortly after the quake. A search for the underlying physics process that might explain these pulses was was undertaken, and one theory (Freund, 2002) demonstrated that charge carriers were released when various types of rocks were stressed in a laboratory environment. It was also significant that the observed charge carrier generation was transient, and resulted in pulsating current patterns. In an attempt to determine if this phenomenon occurred outside of the laboratory environment, the authors scaled up the physics experiment from a relatively small rock sample in a dry laboratory setting, to a large 7 metric tonne boulder comprised of Yosemite granite. This boulder was located in a natural, humid (above ground) setting at Bass Lake, Ca. The boulder was instrumented with two Zonge Engineering, Model ANT4 induction type magnetometers, two Trifield Air Ion Counters, a surface charge detector, a geophone, a Bruker Model EM27 Fourier Transform Infra Red (FTIR) spectrometer with Sterling cycle cooler, and various temperature sensors. The boulder was stressed over about 8 h using expanding concrete (Bustartm), until it fractured into three major pieces. The recorded data showed surface charge build up, magnetic pulsations, impulsive air conductivity changes, and acoustical cues starting about 5 h before the boulder actually broke. These magnetic and air conductivity pulse signatures resembled both the laboratory rock stressing results and the 30 October 2007 M5.4 Alum Rock earthquake field data. The second part of this paper examined other California earthquakes, prior to the Alum Rock earthquake, to see if magnetic pulsations were also present prior to those events. A search for field examples of medium earthquakes was performed to identify earthquakes where functioning magnetometers were present within 20 km, the expected detection range of the magnetometers. Two earthquakes identified in the search included the 12 August 1998 M5.1 San Juan Bautista (Hollister Ca.) earthquake and the 28 September 2004 M6.0 Parkfield Ca. earthquake. Both of these data sets were recorded using EMI Corp. Model BF4 induction magnetometers, installed in equipment owned and operated by UC Berkeley. Unfortunately, no air conductivity or IR data were available for these earthquake examples. This new analysis of old data used the raw time series data (40 samples per s), and examined the data for short duration pulsations that exceeded the normal background noise levels at each site, similar to the technique used at Alum Rock. Analysis of Hollister magnetometer, positioned 2 km from the epicenter, showed a significant increase in magnetic pulsations above quiescient threshold levels several weeks prior, and especially 2 days prior to the quake. The pattern of positive and negative pulsations observed at Hollister, were similar, but not identical to Alum Rock in that the pattern of pulsations were interspersed with Pc 1 pulsation trains, and did not start 2 weeks prior to the quake, but rather 2 days prior. The Parkfield data (magnetometer positioned 19 km from the epicenter) showed much smaller pre-earthquake pulsations, but the area had significantly higher conductivity (which attenuates the signals). More interesting was the fact that significant pulsations occurred between the aftershock sequences of quakes as the crustal stress patterns were migrating. Comparing laboratory, field experiments with a boulder, and earthquake events, striking similarities were noted in magnetic pulsations and air conductivity changes, as well as IR signals (where instrumented). More earthquake samples, taken with the appropriate detectors and within 10–15 km proximity to large (>M5) earthquakes, are still needed to provide more evidence to understand the variability between earthquakes and various electromagnetic signals detected prior to large earthquakes.

Inferring spatiotemporal network patterns from intracranial EEG data

The characterization of spatial network dynamics is desirable for a better understanding of seizure physiology. The goal of this work is to develop a computational method for identifying transient spatial patterns from intracranial electroencephalographic (iEEG) data. Starting with bivariate synchrony measures, such as phase correlation, a two-step clustering procedure is used to identify statistically significant spatial network patterns, whose temporal evolution can be inferred. We refer to this as the composite synchrony profile (CSP) method. The CSP method was verified with simulated data and evaluated using ictal and interictal recordings from three patients with intractable epilepsy. Application of the CSP method to these clinical iEEG datasets revealed a set of distinct CSPs with topographies consistent with medial temporal/limbic and superior parietal/medial frontal networks thought to be involved in the seizure generation process. By combining relatively straightforward multivariate signal processing techniques, such as phase synchrony, with clustering and statistical hypothesis testing, the methods we describe may prove useful for network definition and identification. The network patterns we observe using the CSP method cannot be inferred from direct visual inspection of the raw time series data, nor are they apparent in voltage-based topographic map sequences.

Forecasting method for water quality time series of few and abnormal data

According to the characteristics of small samples and sudden change of some sections in Three Gorges,the ELS-SVM water quality prediction model for Three Gorges was proposed,which consisted of Equal Level Pretreatment Model(ELPM) and Least Squares Support Vector Machines(LS-SVM).ELPM was brought forward to preprocess raw time series data to enhance the smoothness,and Simulated Annealing(SA) algorithm was used for choosing the optimal parameters of LS-SVM.The experimental results indicate that,compared to the typical forecasting models in limited samples prediction,the ELS-SVM forecasting model is superior to the others in water quality parameter prediction of Three Gorges.

A CBR‐BASED, CLOSED‐LOOP ARCHITECTURE FOR TEMPORAL ABSTRACTIONS CONFIGURATION

In the hemodialysis domain, we are implementing a case‐based, closed‐loop architecture aimed at configuring temporal abstractions (TA), which will be applied to time series data. The advantage of a case‐based approach is the one of “quickly” obtaining a suitable TA parameter configuration, simply by looking at the most similar already configured case, where configured cases are indexed by means of contextual information. The retrieved configuration, together with the time series data, is then used as an input to a TA processing module, able to provide a set of qualitative states, trends, and significant combinations of both as an output. TA processing results can finally be evaluated, possibly leading to a (human‐supervized) reorganization/revision of the case base content, to ameliorate future TA configuration sessions—thus closing the loop. The work is being integrated with RHENE, a system for case‐based retrieval in hemodialysis, able to work both on raw time series data and on preprocessed (by means of TA) ones.

Stock time series visualization based on data point importance

Time series visualization is a fundamental task in most financial applications. A framework that can reduce the dimensionality of the time series data, sufficiently accurate so that it can capture the actual shape of the time series but, at the same time, salient points will not be smoothed out can take advantage of charting the raw time series data. On the other hand, it is preferable that the representation framework can handle the multi-resolution problem rather than reduce the dimension to a fixed level only. In this paper, a framework that represents and visualizes time series data based on data point importance is proposed. Furthermore, discovering frequently appearing and surprising patterns are non-trivial tasks in financial applications. A method for discovering patterns across different resolutions is proposed. The proposed method is based on a modified version of VizTree. By converting the time series to symbol string based on data point importance, the potential patterns with different lengths can be encoded in the VizTree for visual pattern discovery while the important points and the overall shape of the time series patterns can be preserved even under a high compression ratio. Various experiments were conducted to evaluate the performance of the proposed framework. One may find it particularly attractive in financial applications like stock data analysis.

A Time Series Weather Radar Simulator Based on High-Resolution Atmospheric Models

Abstract A three-dimensional radar simulator capable of generating simulated raw time series data for a weather radar has been designed and implemented. The characteristics of the radar signals (amplitude, phase) are derived from the atmospheric fields from a high-resolution numerical weather model, although actual measured fields could be used. A field of thousands of scatterers is populated within the field of view of the virtual radar. Reflectivity characteristics of the targets are determined from well-known parameterization schemes. Doppler characteristics are derived by forcing the discrete scatterers to move with the three-dimensional wind field. Conventional moment-generating radar simulators use atmospheric conditions and a set of weighting functions to produce theoretical moment maps, which allow for the study of radar characteristics and limitations given particular configurations. In contrast to these radar simulators, the algorithm presented here is capable of producing sample-to-sample time series data that are collected by a radar system of virtually any design. Thus, this new radar simulator allows for the test and analysis of advanced topics, such as phased array antennas, clutter mitigation schemes, waveform design studies, and spectral-based methods. Limited examples exemplifying the usefulness and flexibility of the simulator will be provided.

1-D electrical conductivity structure beneath the Philippine Sea: Results from an ocean bottom magnetotelluric survey

Eight-months of observation using Ocean Bottom Electro-Magnetometers (OBEMs) have allowed us to estimate the regional electrical conductivity structure beneath the Philippine Sea. Six OBEMs were deployed along a line crossing the Philippine Sea from NW to SE and five of them recorded useful data. The raw time series data were cleaned up before we estimated the magnetotelluric (MT) impedance tensor. Conductivity structure at five sites is estimated using 1-D Occam's inversion to fit the determinant average of each MT impedance tensor after a correction for the effect of topography. We examined effect from two dimensionalities on the 1-D conductivity structure and the robustness of solutions. The results of the 1-D conductivity structural model are strongly related to tectonic setting and the crustal age beneath each site. The structure beneath the spreading axis of the Mariana Trough shows a distinct low conductivity structure at depths of 50–150 km and it probably reflects the upwelling dynamics operating beneath the spreading axis. These low values are comparable with that of olivine with low hydrogen content, implying that (1) the melting process extracts water from minerals such as olivine, and (2) the melt beginning depth in the Mariana Trough is deeper than that of the typical MORB source region. The off-axis conductivity profiles infer the existence of a high conductivity peak or a conductivity gradient change at mid-depth. The depth level of the peak increases with crustal age, suggesting that the conductivity structure is related to a geothermal structure and that these conductivity profiles are explained by the temperature gradient change, possibly combined with the presence of partial melt. Our results suggest that further ocean bottom EM study has high potential to investigate the temperature gradient change and amount of hydrogen (water) and melt in the upper mantle.

Detecting and characterising small changes in urban nitrogen dioxide concentrations

Detecting and quantifying abnormal changes in the concentration of urban air pollutants can be difficult due to the influence of meteorology and atmospheric chemistry. This study presents methods to detect and characterise small changes in the concentration of nitrogen dioxide (NO 2) that deviate from expected behaviour. Generalized additive models (GAMs) are used to describe daily mean NO 2 concentrations at roadside monitoring sites to determine how concentrations deviate from measured concentrations. Structural change methods are applied to time series describing this difference to identify change-points, where concentrations of NO 2 deviate significantly from expected behaviour. Methods are also used which quantify the timing and uncertainty associated in the timing of these change-points. For most time series data considered in London, concentrations of NO 2 underwent relatively abrupt changes rather than smoothly varying increases; changes which remain largely undetected in raw time series data. Most change-points occurred in late 2002 or early 2003, and the factors, which may have contributed to them, are discussed. These methods can also identify technical problems with monitoring equipment and are applicable to other instances where detecting sudden atmospheric composition change is important.

Stochastic optimization for modeling physiological time series: application to the heart rate response to exercise

Stochastic optimization is applied to the problem of optimizing the fit of a model to the time series of raw physiological (heart rate) data. The physiological response to exercise has been recently modeled as a dynamical system. Fitting the model to a set of raw physiological time series data is, however, not a trivial task. For this reason and in order to calculate the optimal values of the parameters of the model, the present study implements the powerful stochastic optimization method ALOPEX IV, an algorithm that has been proven to be fast, effective and easy to implement. The optimal parameters of the model, calculated by the optimization method for the particular athlete, are very important as they characterize the athlete's current condition. The present study applies the ALOPEX IV stochastic optimization to the modeling of a set of heart rate time series data corresponding to different exercises of constant intensity. An analysis of the optimization algorithm, together with an analytic proof of its convergence (in the absence of noise), is also presented.

Oxygen exchange flux between sediment and waterin an intertidal sandflat, measured in situ by the eddy-correlation method

High temporal-resolution fluctuations in oxygen concentration and vertical velocity were measured over an intertidal sandflat (water depth 5 Hz) observed in the vertical velocity and oxygen concentration data made little contribution to the total oxygen flux (0 to 7%). However, trends (steady change over a longer time scale) caused significant artifacts in the estimated fluxes for several cases. Thus, removal of trends from raw time-series data is recommended. The co-spectrum of the fluctuating components of vertical velocity and oxygen con- centration revealed that the oxygen flux at a frequency band between 0.3 and 1.4 Hz (at a period from 0.7 to 3.3 s) was a major contributor to the total oxygen flux. This frequency was consistent with the dominant frequency of vertical velocity, indicating that transport and exchange of porewater and overlying water by wind-induced waves may be crucial processes to dissolved oxygen flux between permeable sandy sediments and shallow waters.

Vehicle classification in distributed sensor networks

The task of classifying the types of moving vehicles in a distributed, wireless sensor network is investigated. Specifically, based on an extensive real world experiment, we have compiled a data set that consists of 820 MByte raw time series data, 70 MByte of pre-processed, extracted spectral feature vectors, and baseline classification results using the maximum likelihood classifier. The purpose of this paper is to detail the data collection procedure, the feature extraction and pre-processing steps, and baseline classifier development. The database is available for download at http://www.ece.wisc.edu/~sensit starting on July 2003.

GPS time series modeling by autoregressive moving average method: Application to the crustal deformation in central Japan

Autoregressive moving average (ARMA) method is applied to modeling the time series of position changes of GPS sites, obtained by the Geographical Survey Institute (GSI) of Japan during the period from April 1996 to March 1998. The present application is focused on denoising of the GPS time series data where only white noise is considered, and detection of data discontinuities and outliers in order to obtain time-averaged velocity and strain fields in central Japan. The data discontinuities are detected by a typical Kalman filter algorithm. The outliers are eliminated by using robust estimation techniques during the ARMA process. The averaged strain field, calculated by the least-squares collocation method from the improved two-year time series data, distinguishes clearly between the tectonically active and inactive regions. Higher maximum shear strain rates were detected in the southern area of the Kanto district. In the areas with very high seismicities, however, the difference between the maximum shear strain rates, that were estimated from the raw time series data and the ARMA-analyzed data, amounted to about 0.2 microstrain/yr. This indicates that the existence of noise and discontinuities can lead to an over-prediction of the strain field.

More effective time-series analysis and forecasting

Our aim is to suggest ways of improving time-domain modelling, for the purpose of more effective forecasting, by better interpretation of the sample autocorrelations and partial autocorrelations obtained from raw time-series data. For this objective, we assume no specialist knowledge, as we start by surveying all those standard ideas of univariate analysis which are needed for the subsequent development of our thesis.