Levels Of Temporal Aggregation Research Articles

Effects of multicollinearity and data granularity on regression models of stream temperature

Water temperature is a key factor influencing biota of stream ecosystems. Hence, it is important to comprehend the environmental drivers of stream temperature for robust prediction of conditions and effective management of stream communities. Linear regression models are commonly used for predictive purposes, but their predictive capacity and interpretability can be significantly affected by their complexity and the structure of input data. In some cases, researchers may be obligated to favor prediction power or interpretability while compromising the other. Therefore, insight into relationships between model fit, correlation among predictor variables (i.e., multicollinearity), and level of temporal aggregation of data (i.e., data granularity) may be helpful to reduce such trade-offs. In this paper, we investigated these relationships within a hierarchical set of multiple linear regression (MLR) models examining environmental factors influencing stream temperature dynamics. Our findings showed that as the number of predictor variables (i.e., model complexity) increased, the magnitude of multicollinearity in MLR models increased, but model fit also increased. The results also revealed that using data averaged over longer time frames (i.e., coarser data granularity) yielded high multicollinearity, as indexed by variance inflation factor values (VIF) for all model predictors. This led to higher variance in parameter estimates (i.e., parameter instability) and potential challenges in model interpretation as the sign of parameter estimates changed in many streams examined. Multicollinearity was not the only reason for these changes in the sign of parameter estimates as they were also observed in simple linear regression models across varying levels of data granularity. Based on our findings, we conclude that the selection of data granularity is an important consideration in multiple regression modeling, with profound implications for model interpretability.

Scale effects and implications of the stochastic structure of customer water demands

ABSTRACT The effect of different temporal (from seconds to months) and spatial aggregation scales (from individual users to full urban areas) on water demand behavior has been explored to a limited degree. The effort described here extends those works by evaluating the scale effects of residential water consumption in a unique US data set that covers 10,000 households with a 1-gallon (3.79 L) hourly resolution over 2 years. A preliminary data analysis and a sequential Principal Component Analysis (PCA) is carried out to assess the effect of different temporal (weekly, daily, hourly) and spatial aggregation (individual meters and groups every 10, 100 and 1,000 meters) levels on demand. Results show that individual users act very differently from each other, and individual consumer variability is only canceled out when a significant number of households are aggregated. The implications of this finding are assessed from a hydraulic modeling perspective as the spatiotemporal scale of measurements may condition the type of analysis that can be carried out in practice. However, additional work is needed to explore the point at which it may be worth embracing a micro (per fixture/household) or a macro (per node/network) approach for different purposes.

Role of mean and variability change in changes in European annual and seasonal extreme precipitation events

Abstract. The frequency of precipitation extremes is set to change in response to a warming climate. Thereby, the change in extreme precipitation event occurrence is influenced by both a shift in the mean and a change in variability. How large the individual contributions are from either of them (mean or variability) to the change in precipitation extremes is largely unknown. This is, however, relevant for a better understanding of how and why climate extremes change. For this study, two sets of forcing experiments from the regional CRCM5 initial-condition large ensemble are used: a set of 50 members with historical and RCP8.5 forcing and a 35-member (700-year) ensemble of pre-industrial natural forcing. The concept of the probability risk ratio is used to partition the change in extreme-event occurrence into contributions from a change in mean climate or a change in variability. The results show that the contributions from a change in variability are in parts equally important to changes in the mean and can even exceed them. The level of contributions shows high spatial variation, which underlines the importance of regional processes for changes in extremes. While over Scandinavia or central Europe the mean influences the increase in extremes more, the increase is driven by changes in variability over France, the Iberian Peninsula, and the Mediterranean. For annual extremes, the differences between the ratios of contribution of mean and variability are smaller, while on seasonal scales the difference in contributions becomes larger. In winter (DJF) the mean contributes more to an increase in extreme events, while in summer (JJA) the change in variability drives the change in extremes. The level of temporal aggregation (3, 24, 72 h) has only a small influence on annual and winterly extremes, while in summer the contribution from variability can increase with longer durations. The level of extremeness for the event definition generally increases the role of variability. These results highlight the need for a better understanding of changes in climate variability to better understand the mechanisms behind changes in climate extremes.

Recent developments in multivariate wind and solar power forecasting

AbstractThe intermittency of renewable energy sources, such as wind and solar, means that they require reliable and accurate forecasts to integrate properly into energy systems. This review introduces and examines a selection of state‐of‐the‐art methods that are applied for multivariate forecasting of wind and solar power production. Methods such as conditional parametric and combined forecasting already see wide use in practice, both commercially and scientifically. In the context of multivariate forecasting, it is vital to model the dependence between forecasts correctly. In recent years, reconciliation of forecasts to ensure coherency across spatial and temporal aggregation levels has shown great promise in increasing the accuracy of renewable energy forecasts. We introduce the methodology used for forecast reconciliation and review some recent applications for wind and solar power forecasting. Many forecasters are beginning to see the benefit of the greater information provided by probabilistic forecasts. We highlight stochastic differential equations as a method for probabilistic forecasting, which can also model the dependence structure. Lastly, we discuss forecast evaluation and how choosing a proper approach to evaluation is vital to avoid misrepresenting forecasts.This article is categorized under: Climate and Environment > Net Zero Planning and Decarbonization Sustainable Energy > Solar Energy Sustainable Energy > Wind Energy

Explaining the Varying Patterns of COVID-19 Deaths Across the United States: 2-Stage Time Series Clustering Framework.

BackgroundSocially vulnerable communities are at increased risk for adverse health outcomes during a pandemic. Although this association has been established for H1N1, Middle East respiratory syndrome (MERS), and COVID-19 outbreaks, understanding the factors influencing the outbreak pattern for different communities remains limited.ObjectiveOur 3 objectives are to determine how many distinct clusters of time series there are for COVID-19 deaths in 3108 contiguous counties in the United States, how the clusters are geographically distributed, and what factors influence the probability of cluster membership.MethodsWe proposed a 2-stage data analytic framework that can account for different levels of temporal aggregation for the pandemic outcomes and community-level predictors. Specifically, we used time-series clustering to identify clusters with similar outcome patterns for the 3108 contiguous US counties. Multinomial logistic regression was used to explain the relationship between community-level predictors and cluster assignment. We analyzed county-level confirmed COVID-19 deaths from Sunday, March 1, 2020, to Saturday, February 27, 2021.ResultsFour distinct patterns of deaths were observed across the contiguous US counties. The multinomial regression model correctly classified 1904 (61.25%) of the counties’ outbreak patterns/clusters.ConclusionsOur results provide evidence that county-level patterns of COVID-19 deaths are different and can be explained in part by social and political predictors.

Seasonality in High Frequency Time Series

Time series observed at higher frequencies than monthly frequency display complex seasonal patterns that result from the combination of multiple seasonal patterns (with annual, monthly, weekly and daily periodicities) and varying periods, due to the irregularity of the calendar. Seasonality in high frequency data is modelled from two main perspectives: the stochastic harmonic approach, based on the Fourier representation of a periodic function, and the time-domain random effects approach. An encompassing representation illustrates the conditions under which they are equivalent. Three major challenges are considered: the first deals with modelling the effect of moving festivals, holidays and other breaks due to the calendar. Secondly, robust estimation and filtering methods are needed to tackle the level of outlier contamination, which is typically high, due to the lower level of temporal aggregation and the raw nature of the data. Finally, model selection strategies play an important role, as the number of harmonic or random components that are needed to account for the complexity of seasonality can be very large.

Aggregation levels when the time between events is Weibull distributed

Practical applications always require an assessment of an aggregation process that is best for early detection of any outbreak of events including sales, warrantee claims or disease outbreaks. This article provides a means of deciding on the level of temporal aggregation that best suits the needs of whoever aims to monitor a process or a service. This article aims to achieve a practical aggregation level for monitoring events when the in-control time between events (TBE) in the targeted process or service follows a non-homogeneous Weibull distribution. We analyze the impact of various aggregation levels on early detection of outbreaks with different magnitudes using several monitoring schemes including an adaptive exponentially weighted moving average (EWMA) plan and several simultaneous EWMA plans with differing amounts of temporal memory for TBE data. We also consider monitoring the related counting processes to address the problem of deciding on the temporal aggregation level. To the best of our knowledge, the effect of various levels of temporal aggregation on detecting outbreaks in TBEs, when they are Weibull distributed, is not studied thoroughly.

Heat load forecasting using adaptive temporal hierarchies

Heat load forecasts are crucial for energy operators in order to optimize the energy production at district heating plants for the coming hours. Furthermore, forecasts of heat load are needed for optimized control of the district heating network since a lower temperature reduces the heat loss, but the required heat supply at the end-users puts a lower limit on the temperature level. Consequently, improving the accuracy of heat load forecasts leads to savings and reduced heat loss by enabling improved control of the network and an optimized production schedule at the plant. This paper proposes the use of temporal hierarchies to enhance the accuracy of heat load forecasts in district heating. Usually, forecasts are only made at the temporal aggregation level that is the most important for the system. However, forecasts for multiple aggregation levels can be reconciled and lead to more accurate forecasts at essentially all aggregation levels. Here it is important that the auto- and cross-covariance between forecast errors at the different aggregation levels are taken into account. This paper suggests a novel framework using temporal hierarchies and adaptive estimation to improve heat load forecast accuracy by optimally combining forecasts from multiple aggregation levels using a reconciliation process. The weights for the reconciliation are computed using an adaptively estimated covariance matrix with a full structure, enabling the process to share time-varying information both within and between aggregation levels. The case study shows that the proposed framework improves the heat load forecast accuracy by 15% compared to commercial state-of-the-art operational forecasts.

Dimensionality reduction in forecasting with temporal hierarchies

Combining forecasts from multiple temporal aggregation levels exploits information differences and mitigates model uncertainty, while reconciliation ensures a unified prediction that supports aligned decisions at different horizons. It can be challenging to estimate the full cross-covariance matrix for a temporal hierarchy, which can easily be of very large dimension, yet it is difficult to know a priori which part of the error structure is most important. To address these issues, we propose to use eigendecomposition for dimensionality reduction when reconciling forecasts to extract as much information as possible from the error structure given the data available. We evaluate the proposed estimator in a simulation study and demonstrate its usefulness through applications to short-term electricity load and financial volatility forecasting. We find that accuracy can be improved uniformly across all aggregation levels, as the estimator achieves state-of-the-art accuracy while being applicable to hierarchies of all sizes.

Seasonality in High Frequency Time Series

Time series observed at higher frequencies than monthly frequency display complex seasonal patterns that result from the combination of multiple seasonal patterns (with annual, monthly, weekly and daily periodicities) and varying periods, due to the irregularity of the calendar. The paper deals with modelling seasonality in high frequency data from two main perspectives: the stochastic harmonic approach, based on the Fourier representation of a periodic function, and the time-domain random effects approach. An encompassing representation illustrates the conditions under which they are equivalent. Three major challenges are considered: the first deals with modelling the effect of moving festivals, holidays and other breaks due to the calendar. Secondly, robust estimation and filtering methods are needed to tackle the level of outlier contamination, which is typically high, due to the lower level of temporal aggregation and the raw nature of the data. Finally, we focus on model selection strategies, which are important, as the number of harmonic or random components that are needed to account for the complexity of seasonality can be very large.

Temporal Aggregation Effects in Spatiotemporal Traffic Modelling.

Spatiotemporal models are a popular tool for urban traffic forecasting, and their correct specification is a challenging task. Temporal aggregation of traffic sensor data series is a critical component of model specification, which determines the spatial structure and affects models’ forecasting accuracy. Through extensive experiments with real-world data, we investigated the effects of the selected temporal aggregation level for forecasting performance of different spatiotemporal model specifications. A set of analysed models include travel-time-based and correlation-based spatially restricted vector autoregressive models, compared to classical univariate and multivariate time series models. Research experiments are executed in several dimensions: temporal aggregation levels, forecasting horizons (one-step and multi-step forecasts), spatial complexity (sequential and complex spatial structures), the spatial restriction approach (unrestricted, travel-time-based and correlation-based), and series transformation (original and detrended traffic volumes). The obtained results demonstrate the crucial role of the temporal aggregation level for identification of the spatiotemporal traffic flow structure and selection of the best model specification. We conclude that the common research practice of an arbitrary selection of the temporal aggregation level could lead to incorrect conclusions on optimal model specification. Thus, we recommend extending the traffic forecasting methodology by validation of existing and newly developed model specifications for different temporal aggregation levels. Additionally, we provide empirical results on the selection of the optimal temporal aggregation level for the discussed spatiotemporal models for different forecasting horizons.

A visual big data system for the prediction of weather-related variables: Jordan-Spain case study

The Meteorology is a field where huge amounts of data are generated, mainly collected by sensors at weather stations, where different variables can be measured. Those data have some particularities such as high volume and dimensionality, the frequent existence of missing values in some stations, and the high correlation between collected variables. In this regard, it is crucial to make use of Big Data and Data Mining techniques to deal with those data and extract useful knowledge from them that can be used, for instance, to predict weather phenomena. In this paper, we propose a visual big data system that is designed to deal with high amounts of weather-related data and lets the user analyze those data to perform predictive tasks over the considered variables (temperature and rainfall). The proposed system collects open data and loads them onto a local NoSQL database fusing them at different levels of temporal and spatial aggregation in order to perform a predictive analysis using univariate and multivariate approaches as well as forecasting based on training data from neighbor stations in cases with high rates of missing values. The system has been assessed in terms of usability and predictive performance, obtaining an overall normalized mean squared error value of 0.00013, and an overall directional symmetry value of nearly 0.84. Our system has been rated positively by a group of experts in the area (all aspects of the system except graphic desing were rated 3 or above in a 1–5 scale). The promising preliminary results obtained demonstrate the validity of our system and invite us to keep working on this area.

Non-Invasive Behavioral Reference Group Categorization Considering Temporal Granularity and Aggregation Level of Energy Use Data

Within residences, normative messaging interventions have been gaining interest as a cost-effective way to promote energy-saving behaviors. Behavioral reference groups are one important factor in determining the effectiveness of normative messages. More personally relevant and meaningful groups are likely to promote behavior change. Using readily available energy-use profiles in a non-invasive manner permits the creation of highly personalized reference groups. Unfortunately, how data granularity (e.g., minute and hour) and aggregation (e.g., one week and one month) affect the performance of energy profile-based reference group categorization is not well understood. This research evaluates reference group categorization performance across different levels of data granularity and aggregation. We conduct a clustering analysis using one-year of energy use data from 2248 households in Holland, Michigan USA. The clustering analysis reveals that using six-hour intervals results in more personalized energy profile-based reference groups compared to using more granular data (e.g., 15 min). This also minimizes computational burdens. Further, aggregating energy-use data over all days of twelve weeks increases the group similarity compared to less aggregated data (e.g., weekdays of twelve weeks). The proposed categorization framework enables interveners to create personalized and scalable normative feedback messages.

Rebuilding depleted fisheries towards BMSY under uncertainty: harvest control rules outperform combined management measures

AbstractRebuilding depleted fisheries towards sustainable levels, such as BMSY, is challenging under uncertainty. Although a substantial amount of research has highlighted the importance of accounting for uncertainty in fisheries management, tactical measures remain to be identified. We consider two approaches to achieve this goal: (i) the naive maximum sustainable yield (MSY) approach, combining management measures based on effort control, catch quotas, and spatial–temporal closures, and (ii) the harvest control rules (HCRs) approach, developing HCRs based on short-term or long-term targets. A suite of strategies is developed accordingly and tested with management strategy evaluation for their performance under four sources of uncertainty that may negatively impact management effects, including reduced recruitment strength, increased natural mortality, inadequate implementation error, and varying levels of temporal effort aggregation. Combining management measures using the naive MSY approach is found to perform poorly in tackling uncertainty. Complex HCRs that account for both short-term and long-term BMSY targets can mitigate the adverse effects of uncertainty. The rebuilding target can be only achieved by compromising yield, especially when uncertainties with natural mortality and recruitment are present. Strategies based on catch quotas are prone to all sources of uncertainty, indicating latent risks in many current management practices.

Predictability of hourly nitrogen dioxide concentration

Temporal aggregation of air quality time series is typically used to investigate stylized facts of the underlying series such as multiple seasonal cycles. While aggregation reduces complexity, commonly used aggregates can suffer from non-representativeness or non-robustness. For example, definitions of specific events such as extremes are subjective and may be prone to data contaminations. The aim of this paper is to assess the predictability of hourly nitrogen dioxide concentrations and to explore how predictability depends on (i) level of temporal aggregation, (ii) hour of day, and (iii) concentration level. Exploratory tools are applied to identify structural patterns, problems related to commonly used aggregate statistics and suitable statistical modeling philosophies, capable of handling multiple seasonalities and non-stationarities. Hourly times series and subseries of daily measurements for each hour of day are used to investigate the predictability of pollutant levels for each hour of day, with prediction horizons ranging from one hour to one week ahead. Predictability is assessed by time series cross validation of a loss function based on out-of-sample prediction errors. Empirical evidence on hourly nitrogen dioxide measurements suggests that predictability strongly depends on conditions (i)-(iii) for all statistical models: for specific hours of day, models based on daily series outperform models based on hourly series, while in general predictability deteriorates with exposure level.

Systemic risk and other interdependencies among banks in Bolivia

Abstract We performed a study of the Bolivian high-value payment system between 2015 and 2020 and of the network of interbank exposures between 2018 and 2020. We characterize the network structure of these systems and perform a systemic risk analysis. We find that the connectivity of the payment network has increased over time, and that it displays a core-periphery structure. Using the SinkRank algorithm, we identify two institutions as the most important throughout the period of study. However, by exploring daily, monthly and yearly networks, we find that, in general, the ranking depends on the level of temporal aggregation. Concerning interbank exposures, we carry out a systemic risk analysis using the DebtRank algorithm. In this analysis, we adjust the banks’ Tier 1 capital to account for credit risk from the commercial loans portfolios, based on the estimation of unexpected losses. We find that average DebtRank and vulnerability of banks tend to increase over time. We link this to a corresponding trend in the size of exposures. We observe spikes in DebtRank and vulnerability, which also occur in correspondence with spikes observed in the size of exposures.

Evaluation of Precipitation Estimates CMORPH-CRT on Regions of Mexico with Different Climates

Satellite-based precipitation (SBP) products with global coverage have the potential to overcome the lack of information in places where there are no rain gauges to perform hydrological analyses; however, it is necessary to evaluate the reliability of the SBP products. In this study, we evaluated the performance of the Climate Prediction Center morphing technique with corrected bias (CMORPH-CRT) product in 14 sites in Mexico. The evaluation was carried out using two approaches: (1) using categorical metrics that include indicators of probability of detection (POD), false alarm rate (FAR), critical success index (CSI), and frequency bias index (FBI); and (2) through statistical indicators such as the mean absolute error (MAE), root mean square error (RMSE), relative bias (RB), and correlation coefficient (CC). The analysis was carried out with two levels of temporal aggregation: 30 min and daily. The results indicate that the CMORPH-CRT product overestimates the number of precipitation events in most cases since FBI values greater than 1 in 78.6% of analyzed stations were obtained. Also, we obtained CC values in the range of 0.018 to 0.625, which implied weak to moderate correlations, and found that in all stations, the CMORPH-CRT product overestimates the precipitation (RB > 0).

Improving freeway segment crash prediction models by including disaggregate speed data from different sources

Traditional traffic safety analyses use highly aggregated data, typically annual average daily traffic (AADT) and annual crash counts. This approach neglects the time-varying nature of critical factors such as traffic speed, volume, and density, and their effects on traffic safety. This paper evaluated the relationship between crashes and quality of flow at different levels of temporal aggregation using continuous count station data and probe data from 4 lane rural freeway and 6 lane urban freeway segments in Virginia. The performance of crash prediction models using traffic and geometric information at 15-minute, hourly, and annual aggregation intervals were contrasted. This study also assessed whether inclusion of speed data improved model performance and examined the effects of using speeds from physical sensors versus speed estimates from private-sector probe speed data. The results showed that using average hourly volume along with average speed and selected geometric variables improved predictions compared to annual models that did not use speed information. When comparing an AADT-based model to an average hourly volume model for total crashes, the mean absolute prediction error improved by 11% for rural models and 20% for urban models. This result was based on volume and speed data from continuous count stations. When private sector probe speed data was used, the rural model performance improved by 10% and urban models by 20%. This trend was consistent for all crash types irrespective of level of injury or number of vehicles involved. Even though models using private sector data performed slightly worse than the ones based on continuous count data, they were still far better than AADT based models. These results indicate that probe based data can be used in developing crash models without harming prediction capability.

Temporal hierarchies with autocorrelation for load forecasting

We propose four different estimators that take into account the autocorrelation structure when reconciling forecasts in a temporal hierarchy. Combining forecasts from multiple temporal aggregation levels exploits information differences and mitigates model uncertainty, while reconciliation ensures a unified prediction that supports aligned decisions at different horizons. In previous studies, weights assigned to the forecasts were given by the structure of the hierarchy or the forecast error variances without considering potential autocorrelation in the forecast errors. Our first estimator considers the autocovariance matrix within each aggregation level. Since this can be difficult to estimate, we propose a second estimator that blends autocorrelation and variance information, but only requires estimation of the first-order autocorrelation coefficient at each aggregation level. Our third and fourth estimators facilitate information sharing between aggregation levels using robust estimates of the cross-correlation matrix and its inverse. We compare the proposed estimators in a simulation study and demonstrate their usefulness through an application to short-term electricity load forecasting in four price areas in Sweden. We find that by taking account of auto- and cross-covariances when reconciling forecasts, accuracy can be significantly improved uniformly across all frequencies and areas.

Effects of the Temporal Aggregation and Meteorological Conditions on the Parameter Robustness of OCO-2 SIF-Based and LUE-Based GPP Models for Croplands

Global retrieval of solar-induced chlorophyll fluorescence (SIF) using remote sensing by means of satellites has been developed rapidly in recent years. Exploring how SIF could improve the characterization of photosynthesis and its role in the land surface carbon cycle has gradually become a very important and active area. However, compared with other gross primary production (GPP) models, the robustness of the parameterization of the SIF model under different circumstances has rarely been investigated. In this study, we examined and compared the effects of temporal aggregation and meteorological conditions on the stability of model parameters for the SIF model ( ε / S I F yield ), the one-leaf light-use efficiency (SL-LUE) model ( ε max ), and the two-leaf LUE (TL-LUE) model ( ε msu and ε msh ). The three models were parameterized based on a maize–wheat rotation eddy-covariance flux tower data in Yucheng, Shandong Province, China by using the Metropolis–Hasting algorithm. The results showed that the values of the ε / S I F yield and ε max were similarly robust and considerably more stable than ε msu and ε msh for all temporal aggregation levels. Under different meteorological conditions, all the parameters showed a certain degree of fluctuation and were most affected at the mid-day scale, followed by the monthly scale and finally at the daily scale. Nonetheless, the averaged coefficient of variation ( C V ) of ε / S I F yield was relatively small (15.0%) and was obviously lower than ε max ( C V = 27.0%), ε msu ( C V = 43.2%), and ε msh ( C V = 53.1%). Furthermore, the SIF model’s performance for estimating GPP was better than that of the SL-LUE model and was comparable to that of the TL-LUE model. This study indicates that, compared with the LUE-based models, the SIF-based model without climate-dependence is a good predictor of GPP and its parameter is more likely to converge for different temporal aggregation levels and under varying environmental restrictions in croplands. We suggest that more flux tower data should be used for further validation of parameter convergence in other vegetation types.