Temporal Data Aggregation Research Articles

Effects of multicollinearity and data granularity on regression models of stream temperature

Water temperature is a key factor influencing biota of stream ecosystems. Hence, it is important to comprehend the environmental drivers of stream temperature for robust prediction of conditions and effective management of stream communities. Linear regression models are commonly used for predictive purposes, but their predictive capacity and interpretability can be significantly affected by their complexity and the structure of input data. In some cases, researchers may be obligated to favor prediction power or interpretability while compromising the other. Therefore, insight into relationships between model fit, correlation among predictor variables (i.e., multicollinearity), and level of temporal aggregation of data (i.e., data granularity) may be helpful to reduce such trade-offs. In this paper, we investigated these relationships within a hierarchical set of multiple linear regression (MLR) models examining environmental factors influencing stream temperature dynamics. Our findings showed that as the number of predictor variables (i.e., model complexity) increased, the magnitude of multicollinearity in MLR models increased, but model fit also increased. The results also revealed that using data averaged over longer time frames (i.e., coarser data granularity) yielded high multicollinearity, as indexed by variance inflation factor values (VIF) for all model predictors. This led to higher variance in parameter estimates (i.e., parameter instability) and potential challenges in model interpretation as the sign of parameter estimates changed in many streams examined. Multicollinearity was not the only reason for these changes in the sign of parameter estimates as they were also observed in simple linear regression models across varying levels of data granularity. Based on our findings, we conclude that the selection of data granularity is an important consideration in multiple regression modeling, with profound implications for model interpretability.

Prospective Spatiotemporal Cluster Detection Using SaTScan: Tutorial for Designing and Fine-Tuning a System to Detect Reportable Communicable Disease Outbreaks.

Staff at public health departments have few training materials to learn how to design and fine-tune systems to quickly detect acute, localized, community-acquired outbreaks of infectious diseases. Since 2014, the Bureau of Communicable Disease at the New York City Department of Health and Mental Hygiene has analyzed reportable communicable diseases daily using SaTScan. SaTScan is a free software that analyzes data using scan statistics, which can detect increasing disease activity without a priori specification of temporal period, geographic location, or size. The Bureau of Communicable Disease's systems have quickly detected outbreaks of salmonellosis, legionellosis, shigellosis, and COVID-19. This tutorial details system design considerations, including geographic and temporal data aggregation, study period length, inclusion criteria, whether to account for population size, network location file setup to account for natural boundaries, probability model (eg, space-time permutation), day-of-week effects, minimum and maximum spatial and temporal cluster sizes, secondary cluster reporting criteria, signaling criteria, and distinguishing new clusters versus ongoing clusters with additional events. We illustrate how to support health equity by minimizing analytic exclusions of patients with reportable diseases (eg, persons experiencing homelessness who are unsheltered) and accounting for purely spatial patterns, such as adjusting nonparametrically for areas with lower access to care and testing for reportable diseases. We describe how to fine-tune the system when the detected clusters are too large to be of interest or when signals of clusters are delayed, missed, too numerous, or false. We demonstrate low-code techniques for automating analyses and interpreting results through built-in features on the user interface (eg, patient line lists, temporal graphs, and dynamic maps), which became newly available with the July 2022 release of SaTScan version 10.1. This tutorial is the first comprehensive resource for health department staff to design and maintain a reportable communicable disease outbreak detection system using SaTScan to catalyze field investigations as well as develop intuition for interpreting results and fine-tuning the system. While our practical experience is limited to monitoring certain reportable diseases in a dense, urban area, we believe that most recommendations are generalizable to other jurisdictions in the United States and internationally. Additional analytic technical support for detecting outbreaks would benefit state, tribal, local, and territorial public health departments and the populations they serve.

The Effect of Temporal Aggregation of Observation Data on Parameter Calibration Results and the Efficiency of Runoff Simulation: Case Study of Small Catchments (Southern Primorskii Krai, Russia)

The Effect of Temporal Aggregation of Observation Data on Parameter Calibration Results and the Efficiency of Runoff Simulation: Case Study of Small Catchments (Southern Primorskii Krai, Russia)

Comparison of methods to aggregate climate data to predict crop yield: an application to soybean

High-dimensional climate data collected on a daily, monthly, or seasonal time step are now commonly used to predict crop yields worldwide with standard statistical models or machine learning models. Since the use of all available individual climate variables generally leads to calculation problems, over-fitting, and over-parameterization, it is necessary to aggregate the climate data used as predictors. However, there is no consensus on the best way to perform this task, and little is known about the impacts of the type of aggregation method used and of the temporal resolution of weather data on model performances. Based on historical data from 1981 to 2016 of soybean yield and climate on 3447 sites worldwide, this study compares different temporal resolutions (daily, monthly, or seasonal) and dimension reduction techniques (principal component analysis (PCA), partial least square regression, and their functional counterparts) to aggregate climate data used as inputs of machine learning and linear regression (LR) models predicting yields. Results showed that random forest models outperformed and were less sensitive to climate aggregation methods than LRs when predicting soybean yields. With our models, the use of daily climate data did not improve predictive performance compared to monthly data. Models based on PCA or averages of monthly data showed better predictive performance compared to those relying on more sophisticated dimension reduction techniques. By highlighting the high sensitivity of projected impact of climate on crop yields to the temporal resolution and aggregation of climate input data, this study reveals that model performances can be improved by choosing the most appropriate time resolution and aggregation techniques. Practical recommendations are formulated in this article based on our results.

Investigation of winter weather crash injury severity using winter storm classification techniques

This study employs a unique methodology for analyzing seasonal traffic crashes by first identifying adverse winter-weather events using traditional traffic-crash data (from Nebraska) and correlating them to detailed meteorological data for subsequent analyses. Specifically, the novel component of this analysis utilized a state-specific winter storm and seasonal severity index metric in conjunction with classifications of storm type. Both crash and meteorological data were combined to determine if the amount of snowfall and/or storm intensity/severity could be a predictor of the number and severity of crashes. Analysis was performed to identify weather conditions causing highway safety concerns. The important takeaway of the study was that most winter weather-related vehicular crashes in Nebraska were associated with relatively minor winter weather conditions. The reported crashes typically occurred with relatively low snowfall amounts due to residual snowfall on the ground, emphasizing the importance of continuing winter maintenance operations activities and public service announcements long after a storm has passed through the region. Modeling crash injury severity revealed that icy pavements, higher duration of snowfall, work zones, and higher number of trucks and buses involved in a crash were linked to higher injury severity; higher visibility was associated with a greater chance of crashes involving visible injuries but a lower likelihood of crashes involving disabling injury/fatality. Snowfall was associated with more visible injury crashes, whereas snow-depth was linked to fewer visible injury and disabling injury/fatality crashes. The study's limitations include the spatial and temporal aggregation of weather data and a lack of detailed winter roadway maintenance/clearance activity data.

The effect of temporal data aggregation to assess the impact of changing temperatures in Europe: an epidemiological modelling study

Daily time-series regression models are commonly used to estimate the lagged nonlinear relation between temperature and mortality. A major impediment to this type of analysis is the restricted access to daily health records. The use of weekly and monthly data represents a possible solution unexplored to date. We temporally aggregated daily temperatures and mortality records from 147 contiguous regions in 16 European countries, representing their entire population of over 400 million people. We estimated temperature-lag-mortality relationships by using standard time-series quasi-Poisson regression models applied to daily data, and compared the results with those obtained with different degrees of temporal aggregation. We observed progressively larger differences in the epidemiological estimates with the degree of temporal data aggregation. The daily data model estimated an annual cold and heat-related mortality of 290,104 (213,745-359,636) and 39,434 (30,782-47,084) deaths, respectively, and the weekly model underestimated these numbers by 8.56% and 21.56%. Importantly, differences were systematically smaller during extreme cold and heat periods, such as the summer of 2003, with an underestimation of only 4.62% in the weekly data model. We applied this framework to infer that the heat-related mortality burden during the year 2022 in Europe may have exceeded the 70,000 deaths. The present work represents a first reference study validating the use of weekly time series as an approximation to the short-term effects of cold and heat on human mortality. This approach can be adopted to complement access-restricted data networks, and facilitate data access for research, translation and policy-making. The study was supported by the ERC Consolidator Grant EARLY-ADAPT (https://www.early-adapt.eu/), and the ERC Proof-of-Concept Grants HHS-EWS and FORECAST-AIR.

Rockets and Feathers Revisited: Asymmetric Retail Gasoline Pricing in the Era of Market Transparency

In this paper, we revisit the empirical observation that prices rise like rockets when input costs increase but fall like feathers when input costs decrease. The analysis draws on a novel data set that includes daily retail prices of gasoline from 12,804 stations in Germany from January 1, 2014 to December 31, 2018. Our findings based on pooled-panel asymmetric error correction models indicate that the pattern of rockets and feathers is the norm rather than the exception. Our results further show that temporal aggregation of station-level price data leads to inaccurate inferences and could account for the inconclusive findings in the literature.

Improved representativeness of simulated climate using natural units and monthly resolution

There is a considerable discrepancy between the temporal and spatial resolution required by climate impact researchers, policy makers, and adaptation planners on the one hand and climate data providers on the other hand. While the spatial and temporal aggregation of climate data is necessary to increase the reliability and robustness of climate information, this often counteracts or even prohibits their use in adaptation planning. The problem is twofold (i.e., space and time) and needs to be approached accordingly. Climate impact research and adaptation planning are the domain of impact experts, politicians, and planners, rather than climate experts. Thus, besides the spatial and temporal resolution, information also needs to be provided on platforms and in data formats that are easily accessible, easy to handle, and easy to understand. We discuss possible steps toward bridging the gap using an example from the federal state Hesse (Germany) as illustration. We aggregate the climate data at a level of “natural units” and provide them as monthly data. We discuss the pros and cons of this kind of processed data for impact research and decision making. The spatial aggregation to “natural units” delivers suitable spatial aggregation, while maintaining physical geographic structures and their climatic characteristics. Within these “natural units,” single grid cell values are usable for climate impact analyses or decision making. The temporal resolution is monthly values, i.e., deviations of single month values for the scenario period from climatological monthly values in the (simulated) reference period. This resolution allows analyzing compound events or consecutive events on a monthly scale within a climatological (30-year) period.

Impacts of Rainfall Data Aggregation Time on Pluvial Flood Hazard in Urban Watersheds

Pluvial floods occur when heavy rainstorms cause the surcharge of the sewer network drainage, representing one of the most impacting natural hazard in urban watersheds. Pluvial flood hazard is usually assessed considering the effect of annual maxima rainfall of short duration, comparable with the typical concentration times of small urban watersheds. However, short duration annual maxima can be affected by an error of underestimation due to the time resolution as well as the aggregation time of the rainfall time series. This study aims at determining the impact of rainfall data aggregation on pluvial flood hazard assessment. Tuscany region (Central Italy) is selected as a case study to perform the assessment of the annual maxima rainfall underestimation error, since the entire region has the same temporal aggregation of rainfall data. Pluvial flood hazard is then evaluated for an urban watershed in the city of Florence (Tuscany) comparing the results obtained using observed (uncorrected) and corrected annual maxima rainfall as meteorological forcing. The results show how the design of rainfall events with a duration of 30 min or shorter is significantly affected by the temporal aggregation, highlighting the importance of correcting annual maxima rainfall for a proper pluvial flood hazard evaluation.

Which Aggregation Fits Best? The Use of Linear Regression to Show the Influence of Temporal and Spatial Aggregation of Sparse Probe Vehicle Data on the Explanation of Travel Time Reliability

Aggregation of sparse probe vehicle data (PVD) is a crucial issue in travel time reliability (TTR) analysis. This study, therefore, examines the effect of temporal and spatial aggregation of sparse PVD on the results of a linear regression analysis where two different measures of TTR are analyzed as the dependent variable. Our results show that by aggregating the data to longer time intervals and coarser spatial units the linear model can explain a higher proportion of the variance in TTR. Furthermore, we find that the effects of road design characteristics in particular depend on the variable used to represent TTR. We conclude that the temporal and spatial aggregation of sparse PVD affects the results of linear regression explaining TTR.

The predictive content of oil price and volatility: New evidence on exchange rate forecasting

We propose oil price volatility as a new fundamental for exchange rate forecasting. This fundamental parallels recent theoretical developments in international finance to resolve the disconnect between macro variables and exchange rates. By focusing on small open economies that export oil, we then provide a comprehensive empirical evaluation of the forecasting performance of the fundamental in multiple countries. We find that oil price and volatility contain out-of-sample predictive content for the exchange rate at daily frequency, especially when crude oil represents a significant fraction of the country’s exports and when there are strong trade ties between the currency areas. We also discover that directional change, i.e. appreciation or depreciation, can be forecasted even in the absence of point forecastability. Our findings are robust to forecast evaluation method, volatility measure and evaluation period, thus supporting the role of financial channels in exchange rate determination. Moreover, the predictive content dissipates when we move from daily to monthly frequency, suggesting that temporal data aggregation may explain the failure to find predictability in past work using monthly or quarterly data.

Quantification of the Information Loss Resulting from Temporal Aggregation of Wind Turbine Operating Data

SCADA operating data are more and more used across the wind energy domain, both as a basis for power output prediction and turbine health status monitoring. Current industry practice to work with this data is by aggregating the signals at coarse resolution of typically 10-min averages, in order to reduce data transmission and storage costs. However, aggregation, i.e., downsampling, induces an inevitable loss of information and is one of the main causes of skepticism towards the use of SCADA operating data to model complex systems such as wind turbines. This research aims to quantify the amount of information that is lost due to this downsampling of SCADA operating data and characterize it with respect to the external factors that might influence it. The issue of information loss is framed by three key questions addressing effects on the local and global scale as well as the influence of external conditions. Moreover, recommendations both for wind farm operators and researchers are provided with the aim to improve the information content. We present a methodology to determine the ideal signal resolution that minimized storage footprint, while guaranteeing high quality of the signal. Data related to the wind, electrical signals, and temperatures of the gearbox resulted as the critical signals that are largely affected by an information loss upon aggregation and turned out to be best recorded and stored at high resolutions. All analyses were carried out using more than one year of 1 Hz SCADA data of onshore wind farm counting 12 turbines located in the UK.

Updated understanding of the deformation characteristics of the Checkerboard Creek rock slope through GB-InSAR monitoring

Ground-based, interferometric, synthetic aperture radar (GB-InSAR) has been successfully implemented for the monitoring and assessment of both natural and man-made slopes. Its advantages include high-frequency data acquisition and high spatial density, providing maps of displacement. The implementation of this tool for civil projects in remote areas and under severe weather conditions present several challenges. This paper presents the implementation of GB-InSAR, particularly using discrete monitoring campaigns and temporal aggregation of data, for resolving logistical issues associated with power availability. This was implemented to monitor a very slow-moving (average displacement rate of ~10 mm/y) 2 to 3 million m3 bedrock landslide known as The Checkerboard Creek Rock Slope located within the Revelstoke dam reservoir, BC, Canada. GB-InSAR deployment has been successful as it confirmed the toppling-like deformation mode, improved definition of the most active area within the exposed slope cut, improved understanding of the seasonality observed in some instruments at the site, and identification of a potential unstable block north of the known active area. Further, the findings in this paper verify the feasibility of implementing GB-InSAR technology to monitor similar slopes associated with the safety of civil infrastructure and in challenging conditions characteristic of the Canadian environment.

Time-Aggregation-Based Lossless Video Encoding for Neuromorphic Vision Sensor Data

Dynamic vision sensors (DVSs) are emerging neuromorphic visual capturing devices, with great advantages in terms of low-power consumption, wide dynamic range, and high temporal resolution in diverse applications, such as autonomous driving, robotics, tactile sensing, and drones. The capturing method results in lower data rates than conventional video. Still, such data can be further compressed. Recent research has shown great benefits of temporal data aggregation on event-based vision data utilization. According to recent results, time aggregation of DVS data not only reduces the data rate but improves classification and object detection accuracy. In this work, we propose a compression strategy, time-aggregation-based lossless video encoding for neuromorphic vision sensor data (TALVEN), which utilizes temporal data aggregation, arrangement of the data in a specific format and lossless video encoding techniques to achieve high compression ratios. The detailed experimental analysis on outdoor and indoor datasets shows that our proposed strategy achieves superior compression ratios than the best state-of-the-art strategies.

Cyclists' noise exposure in a Brazilian medium-sized city.

This study aimed to assess cyclists' exposure to noise in a medium-sized Brazilian city. Mobile sensors were used to conduct noise measurements in streets with and without dedicated cycling infrastructures. The method can be summarized in the following procedures: i) characterization of the study area; ii) data collection and validation; iii) calculation of exposure indicators; and iv) comparison and representation of the results on maps. Two strategies were adopted for the analysis, namely, spatial data aggregation and temporal data aggregation. Thus, measurements were initially organized in 1,200 nodes distributed along the paths. The results indicate that bicycle riders in São Carlos may be exposed, in some routes, to a high proportion of high noise-level segments. In the two routes selected for this study, the cyclist was exposed to noise levels above the adopted threshold (> 75dBA) in 33.2% and 18.9% of the nodes. Also, the possibility of simultaneously working with two related indicators has broadened the classification criteria of the route segments regarding noise exposure.

Review on Statistical Modeling of Travel Time Variability for Road-Based Public Transport

Accurately modeling travel time of road-based public transport can help directly improve current passenger service and operating efficiency. Moreover, it paves the way for control of future high technology automated vehicles, which will share the same characteristics of sharing the road infrastructure with other vehicles; carrying multiple passengers, i.e. having a non-negligible dwell process, and run not completely demand-responsive but in general following a schedule or a target frequency. Recent advances in sensor and communications technology, leading eventually to comprehensive vehicle connectivity, have significantly increased the amount and quality of travel time data available making it possible to better model distributions of travel time of current buses. We assume that the choice of those distributions with regards to transport performance will hold also in the near future. This paper explains definitions of travel time components and explains how they contribute to variability. It focuses on the description of day-to-day variability and systematically reviews the current state of the art for statistically modeling bus travel, running, and dwell time distributions. It considers statistical distributions developed based on empirical data from the research literature. Statistical distributions are powerful tools, as they can describe the inherent variability in data with a limited number of parameters. The review finds that both spatial and temporal data aggregation have an important influence on the statistics as well as the choice of the most appropriate probability distribution. This influence is still not well understood and remains a question for further studies. Furthermore, the review finds that mixture distributions provide good fitting performance, however, it is important to improve the description of components in such distributions, in order to get meaningful and understandable distributions. The methodologies for fitting distributions, for proving if a distribution is suited, and for identifying best fitting, robust and reproducible distribution should be reconsidered. Such a distribution will enable reporting, controlling operations, and disseminating information to operators and travelers. Finally, this review proposes directions for further work.

Use of the CUSUM and Shewhart control chart methods to identify changes of public health significance using childhood blood lead surveillance data.

Local, state, and national childhood blood lead surveillance is based on healthcare providers and clinical laboratories reporting test results to public health departments. Increased interest in detecting blood lead level (BLL) patterns and changes of potential public health significance in a timely manner has highlighted the need for surveillance systems to rapidly detect and investigate these events. Decrease the time to detect changes in surveillance patterns by using an alerting algorithm developed and assessed through historical child blood lead surveillance data analysis. We applied geographic and temporal data-aggregation strategies on childhood blood lead surveillance data and developed a novel alerting algorithm. The alerting algorithm employed a modified cumulative summary/Shewhart algorithm, initially applied on 113 months of data from two jurisdictions with a known increase in the proportion of children <6 years of age with BLLs =>5 µg/dl. Alert signals retrospectively identified time periods in two jurisdictions where a known change in the proportion of children <6 years of age with BLLs >=5 µg/dl occurred. Additionally, we identified alert signals among six of the 18 (33%) randomly selected counties assessed where no previously known or suspected pattern changes existed. The modified cumulative summary/Shewhart algorithm provides a framework for enhanced blood lead surveillance by identifying changes in the proportion of children with BLLs >=5 µg/dl. The algorithm has the potential to alert public health officials to changes requiring further important public health investigation.

Effects of Spatial and Temporal Data Aggregation on the Performance of the Multi‐Radar Multi‐Sensor System

AbstractThe objectives of this study were to (1) evaluate the performance of the Multi‐Radar Multi‐Sensor (MRMS) system in capturing precipitation compared to gauge data, and (2) assess the effects of spatial (1–50 km) and temporal (15–120 min) data aggregation scales on the performance of the MRMS system. Point‐to‐grid comparisons were conducted between 215 rain gauges and the MRMS system. The MRMS system at 1 km spatial and 15 min temporal resolutions captured precipitation reasonably well with average R2, root mean square error (RMSE), and percent bias (PBIAS) values of 0.65, 0.5 mm, and 11.9 mm; whereas Threat Score, probability of detection, and false alarm ratio were 0.57, 0.92, and 0.40, respectively. Decreasing temporal resolution from 15 min to two hours resulted in an increase in R2 and a decrease in RMSE, whereas PBIAS was not affected. Reducing spatial resolution from 1 to 50 km resulted in increases in R2 and PBIAS, whereas RMSE was decreased. Increasing spatial aggregation scale from 1 to 50 km resulted in an R2 increase of only 0.08. Similarly, improvement in R2 was only modest (0.17) compared to an eightfold reduction in temporal resolution (from 15 min to two hours). While aggregating data at coarser temporal resolutions resolved some of the under/overestimation issues of the MRMS system, it was apparent even at coarser spatial and temporal resolutions the MRMS system inherently overestimated smaller precipitation events while underestimated bigger events.

Energy-efficient scheduling scheme with spatial and temporal aggregation for small and massive transmissions in LTE-M networks

Machine-to-machine (M2M) communication is one of the key technologies to realize Internet of Things (IoT). Since IoT applications are mainly for smart sensing, such as metering, home surveillance, disaster detection, and e-health, their special sensing/uploading behaviors will result in periodic and/or event-driven small data transmissions, which may potentially decrease the radio resource efficiency. On the other hand, due to the frequent communication nature of sensing data for IoT applications, the power consumption is increasing dramatically. To reduce the power consumption of IoT devices, the 3rd Generation Partnership Project (3GPP) has defined the discontinuous reception/discontinuous transmission (DRX/DTX) mechanism to allow devices to turn off their radio interfaces and go to sleep in various patterns. However, how to optimize the DRX/DTX scheduling while improving the resource efficiency is still an open issue. In this paper, we investigate an uplink resource allocation problem over long-term evolution machine-to-machine (LTE-M) networks, which is standardized by 3GPP to improve performance on IoT. In this network, we consider the periodic, event-driven, and query-based IoT traffic while minimizing the devices’ power consumption. We prove this problem to be NP-complete and propose an aggregation-efficient DRX/DTX scheduling (AEDS) scheme. This scheme takes advantage of both spatial and temporal data aggregation while applying DRX/DTX for energy saving. Specifically, the scheme consists of three phases. The first phase exploits long-term static scheduling for periodic data to ensure the latency and data rate while minimizing the devices’ wake-up time. The second phase tries to decrease devices’ power consumption through precisely determining their DRX/DTX configurations. Finally, the third phase employs short-term dynamic scheduling for event-driven and query-based data to improve transmission efficiency. Therefore, both small data and power consumption problems are relieved. Extensive simulation results show that the proposed scheme can improve resource efficiency, enlarge network capacity while reducing power consumption compared to the existing schemes.

Visual analysis of traffic data via spatio-temporal graphs and interactive topic modeling

Analyzing of traffic data is an important task for urban planners and managers. Traffic data have the spatio-temporal characteristics, which can reflect the variation of the presence of vehicle in different places over time as well as traffic flow dynamics among different places. The analysis of large-scale GPS trajectory data is a very challenging research due to the complexity of data and the need to extract useful information under cover in data. In this study, we combine temporal and geospatial aggregation of traffic data for obtaining key areas and creating legible traffic flow maps; meanwhile, we make full use of the topic model to capture latent semantic information. Nevertheless, most of the topic models always encounter the plague of choosing the optimal number of topics and cannot easily incorporate numerous types of user feedback. To tackle these problems, we propose an interactive topic modeling equipped with various interactive capabilities which empowers users to explore data from different levels of detail. Finally, we design and implement an interactive visual analytics prototype system based on the spatio-temporal graphs and the interactive topic modeling. The feasibility and validity of our system is demonstrated by conducting two case studies with a real-world traffic data in Hangzhou.