Multiple Aggregation Levels Research Articles

MVPipe: Enabling Lightweight Updates and Fast Convergence in Hierarchical Heavy Hitter Detection

Finding hierarchical heavy hitters (HHHs) (i.e., hierarchical aggregates with exceptionally huge amounts of traffic) is critical to network management, yet it is often challenged by the requirements of fast packet processing, real-time and accurate detection, as well as resource efficiency. Existing HHH detection schemes either incur expensive packet updates for multiple aggregation levels in the IP address hierarchy, or need to process sufficient packets to converge to the required detection accuracy. We present, an invertible sketch that achieves both lightweight updates and fast convergence in HHH detection. builds on the skewness property of IP traffic to process packets via a pipeline of majority voting executions, such that most packets can be updated for only one or few aggregation levels in the IP address hierarchy. We show how can be feasibly deployed in P4-based programmable switches subject to limited switch resources. We also theoretically analyze the accuracy and coverage properties of . Evaluation with real-world Internet traces shows that achieves high accuracy, high throughput, and fast convergence compared to six state-of-the-art HHH detection schemes. It also incurs low resource overhead in the Tofino switch deployment.

Estimating soil organic carbon stock change at multiple scales using machine learning and multivariate geostatistics

Many national and international initiatives rely on spatially explicit information on soil organic carbon (SOC) stock change at multiple scales to support policies aiming at land degradation neutrality and climate change mitigation. In this study, we used regression cokriging with random forest and spatial stochastic cosimulation to predict the SOC stock change between two years (i.e. 1992 and 2010) in Hungary at multiple aggregation levels (i.e. point support, 1 × 1 km, 10 × 10 km square blocks, Hungarian counties and entire Hungary). We also quantified the uncertainty associated with these predictions in order to identify and delimit areas with statistically significant SOC stock change. Our study highlighted that prediction of spatial totals and averages with quantified uncertainty requires a geostatistical approach and cannot be solved by machine learning alone, because it does not account for spatial correlation in prediction errors. The total topsoil SOC stock for Hungary was predicted to increase between 1992 and 2010 with 14.9Tg, with lower and upper limits of a 90% prediction interval equal to 11.2Tg and 18.2Tg, respectively. Results also showed that both the predictions and uncertainties of the average SOC stock change were smaller for larger spatial supports, while spatial aggregation also made it easier to obtain statistically significant SOC stock changes. The latter is important for carbon accounting studies that need to prove in Measurement, Reporting and Verification protocols that observed SOC stock changes are not only practically but also statistically significant.

Heat load forecasting using adaptive temporal hierarchies

Heat load forecasts are crucial for energy operators in order to optimize the energy production at district heating plants for the coming hours. Furthermore, forecasts of heat load are needed for optimized control of the district heating network since a lower temperature reduces the heat loss, but the required heat supply at the end-users puts a lower limit on the temperature level. Consequently, improving the accuracy of heat load forecasts leads to savings and reduced heat loss by enabling improved control of the network and an optimized production schedule at the plant. This paper proposes the use of temporal hierarchies to enhance the accuracy of heat load forecasts in district heating. Usually, forecasts are only made at the temporal aggregation level that is the most important for the system. However, forecasts for multiple aggregation levels can be reconciled and lead to more accurate forecasts at essentially all aggregation levels. Here it is important that the auto- and cross-covariance between forecast errors at the different aggregation levels are taken into account. This paper suggests a novel framework using temporal hierarchies and adaptive estimation to improve heat load forecast accuracy by optimally combining forecasts from multiple aggregation levels using a reconciliation process. The weights for the reconciliation are computed using an adaptively estimated covariance matrix with a full structure, enabling the process to share time-varying information both within and between aggregation levels. The case study shows that the proposed framework improves the heat load forecast accuracy by 15% compared to commercial state-of-the-art operational forecasts.

Joint In-Season and Out-of-Season Promotion Demand Forecasting in a Retail Environment

Inaccurate forecasts of demand during promotions diminish the already meager profit margins of retailers. No forecasting method described in the literature can accurately account for the combination of seasonal sales variations and promotion-induced sales peaks over forecasting horizons of several weeks or months. We address this research gap by developing a forecasting method for seasonal, frequently promoted products that generates accurate predictions, can handle a large number of sales series, and requires minimal training data. In our method's first stage, we forecast the seasonal sales cycle by fitting a harmonic regression model to a decomposed training set, which excludes promotional and holiday sales, and then extrapolate that model to a testing set. In the second stage, we integrate the resulting seasonal forecast into a multiplicative demand function that accounts for consumer stockpiling and captures promotional and holiday sales uplifts. The final model is then fitted using ridge regression. We use sales data from a grocery retailing chain to compare the forecasting accuracy of our method with popular seasonal and promotion demand forecasting models at multiple aggregation levels for both short and long forecasting horizons. The significantly more accurate forecasts generated by our model attest to the merit of the approach developed here.

A rapidly deployed, interactive, online visualization system to support fatality management during the coronavirus disease 2019 (COVID-19) pandemic.

ObjectiveTo create an online visualization to support fatality management in North Carolina.Materials and MethodsA web application aggregates online datasets for coronavirus disease 2019 (COVID-19) infection rates and morgue utilization. The data are visualized through an interactive, online dashboard.ResultsThe web application was shared with state and local public health officials across North Carolina. Users could adjust interactive maps and other statistical charts to view live reports of metrics at multiple aggregation levels (eg, county or region). The application also provides access to detailed tabular data for individual facilities.DiscussionStakeholders found this tool helpful for providing situational awareness of capacity, hotspots, and utilization fluctuations. Timely reporting of facility and county data were key, and future work can help streamline the data collection process. There is potential to generalize the technology to other use cases.ConclusionsThis dashboard facilitates fatality management by visualizing county and regional aggregate statistics in North Carolina.

Towards Systems-Oriented Energy Solutions: A Multilevel Analysis of a Low-Income Energy Efficiency Program in Brazil

There is a consensus among scholars and practitioners that energy solutions, such as electricity services and related products and systems, are paramount to the ability of nations to overcome environmental and social issues. As a result, policymakers and problem solvers in emerging economies have shown a keen interest in the transition to sustainable energy systems. Nevertheless, the design of sustainable energy solutions in low-income markets presents many challenges, such as those related to limited financial resources and poor infrastructure. In low-income markets, the adoption of a systems-oriented approach to product–service combinations may represent a promising alternative to traditional design approaches and result in a more socially and environmentally sound path to economic development. Building on design theory grounded in systems theory, this paper analyzes multiple aggregation levels of the sociotechnical system of a low-income energy efficiency program in Brazil. In this study, the authors examined findings from the literature, carried out field observations, and had discussions with practitioners and experts. The study identifies constraints that hinder energy solutions that could achieve higher levels of socioeconomic and environmental benefits in low-income energy markets. Based on the findings, the paper provides insights into sustainable energy transitions and concludes that low-income energy efficiency programs can be improved through design-led policy and stakeholder collaboration.

Comparing electricity consumption trends: A multilevel index decomposition analysis of the Genevan and Swiss economy

Switzerland, including the canton of Geneva, aims to reduce its electricity consumption following its decision to phase out nuclear electricity production. To investigate whether national policies and a regional programme, both of which aim at improving electricity efficiency, may have had an effect, we disentangle the effects of changes in economic structure, overall economic activity and structure-corrected energy intensity (SCEI) on the electricity consumption in the canton of Geneva and in Switzerland on multiple aggregation levels. The primary sector being negligibly small, we define the economy as the secondary and tertiary sector.Our analysis shows that changes in electricity consumption in the Genevan and Swiss economy were mainly caused by changes in activity and SCEI, although structural changes were not negligible. Specifically, we have shown that correcting for structural change may significantly impact comparisons between regional SCEI trends.Our analysis shows that economy-wide electric energy efficiency improved both in Geneva and Switzerland. This is the case both for the time period 2000–2007 (first period, before implementation of the energy efficiency programme [EEP] éco21 in Geneva) and for the period 2008–2014 (second period, after implementation of the EEP, excluding 2015). In Switzerland, the average reductions in SCEI in the study periods were similar: 1.2% per year in the first period and 1.0% per year in the second period. The average reduction in SCEI in Geneva was faster in the second period (2.6% per year) than in the first period (1.5% per year). These findings suggest that national energy efficiency policies that have been in place since 2001 have been effective.A strong increase in SCEI by 4.6% occurred in Geneva in 2015, diminishing the gap between the Genevan and Swiss SCEI. We found that this increase was likely to be caused by weather effects. When including the year 2015, the average reduction in SCEI in the second period was 1.7% for Geneva and 1.1% for Switzerland. Moreover, we have shown that weather effects were more strongly correlated with SCEI than with absolute electricity consumption, highlighting the importance of correcting for structure and weather effects.Further, weather effects made it difficult to identify contributions from the EEP to the reduction of the Genevan SCEI. We found that the applicability IDA for tracing the effectiveness of EEPs needs to be critically assessed in view of data availability, confounding influences such as weather effects as well as the maturity and size of the EEP.

Demand forecasting by temporal aggregation: Using optimal or multiple aggregation levels?

Recent advances have demonstrated the benefits of temporal aggregation for demand forecasting, including increased accuracy, improved stock control and reduced modelling uncertainty. With temporal aggregation a series is transformed, strengthening or attenuating different elements and thereby enabling better identification of the time series structure. Two different schools of thought have emerged. The first focuses on identifying a single optimal temporal aggregation level at which a forecasting model maximises its accuracy. In contrast, the second approach fits multiple models at multiple levels, each capable of capturing different features of the data. Both approaches have their merits, but so far they have been investigated in isolation. We compare and contrast them from a theoretical and an empirical perspective, discussing the merits of each, comparing the realised accuracy gains under different experimental setups, as well as the implications for business practice. We provide suggestions when to use each for maximising demand forecasting gains.

Computing multiple aggregation levels and contextual features for road facilities recognition using mobile laser scanning data

In recent years, updating the inventory of road infrastructures based on field work is labor intensive, time consuming, and costly. Fortunately, vehicle-based mobile laser scanning (MLS) systems provide an efficient solution to rapidly capture three-dimensional (3D) point clouds of road environments with high flexibility and precision. However, robust recognition of road facilities from huge volumes of 3D point clouds is still a challenging issue because of complicated and incomplete structures, occlusions and varied point densities. Most existing methods utilize point or object based features to recognize object candidates, and can only extract limited types of objects with a relatively low recognition rate, especially for incomplete and small objects. To overcome these drawbacks, this paper proposes a semantic labeling framework by combing multiple aggregation levels (point-segment-object) of features and contextual features to recognize road facilities, such as road surfaces, road boundaries, buildings, guardrails, street lamps, traffic signs, roadside-trees, power lines, and cars, for highway infrastructure inventory. The proposed method first identifies ground and non-ground points, and extracts road surfaces facilities from ground points. Non-ground points are segmented into individual candidate objects based on the proposed multi-rule region growing method. Then, the multiple aggregation levels of features and the contextual features (relative positions, relative directions, and spatial patterns) associated with each candidate object are calculated and fed into a SVM classifier to label the corresponding candidate object. The recognition performance of combining multiple aggregation levels and contextual features was compared with single level (point, segment, or object) based features using large-scale highway scene point clouds. Comparative studies demonstrated that the proposed semantic labeling framework significantly improves road facilities recognition precision (90.6%) and recall (91.2%), particularly for incomplete and small objects.

Leveraging Zipf's law for traffic offloading

Internet traffic has Zipf-like properties at multiple aggregation levels. These properties suggest the possibility of offloading most of the traffic from a complex controller (e.g., a software router) to a simple forwarder (e.g., a commodity switch), by letting the forwarder handle a very limited set of flows; the heavy hitters. As the volume of traffic from a set of flows is highly dynamic, maintaining a reliable set of heavy hitters over time is challenging. This is especially true when we face a volume limit in the non-offloaded traffic in combination with a constraint in the size of the heavy hitter set or its rate of change. We propose a set selection strategy that takes advantage of the properties of heavy hitters at different time scales. Based on real Internet traffic traces, we show that our strategy is able to offload most of the traffic while limiting the rate of change of the heavy hitter set, suggesting the feasibility of alternative router designs.

A Sensitivity Analysis of the Social Vulnerability Index

The Social Vulnerability Index (SoVI), created by Cutter et al. (2003), examined the spatial patterns of social vulnerability to natural hazards at the county level in the United States in order to describe and understand the social burdens of risk. The purpose of this article is to examine the sensitivity of quantitative features underlying the SoVI approach to changes in its construction, the scale at which it is applied, the set of variables used, and to various geographic contexts. First, the SoVI was calculated for multiple aggregation levels in the State of South Carolina and with a subset of the original variables to determine the impact of scalar and variable changes on index construction. Second, to test the sensitivity of the algorithm to changes in construction, and to determine if that sensitivity was constant in various geographic contexts, census data were collected at a submetropolitan level for three study sites: Charleston, SC; Los Angeles, CA; and New Orleans, LA. Fifty-four unique variations of the SoVI were calculated for each study area and evaluated using factorial analysis. These results were then compared across study areas to evaluate the impact of changing geographic context. While decreases in the scale of aggregation were found to result in decreases in the variance explained by principal components analysis (PCA), and in increases in the variance of the resulting index values, the subjective interpretations yielded from the SoVI remained fairly stable. The algorithm's sensitivity to certain changes in index construction differed somewhat among the study areas. Understanding the impacts of changes in index construction and scale are crucial in increasing user confidence in metrics designed to represent the extremely complex phenomenon of social vulnerability.

Automated verification of disaster plans in incident management

PurposeThe purpose of this paper is to create a formal specification language for disaster plans in order to remove possible inconsistencies between disaster plans, and to enable the automated verification of properties from such plans against logs of actual incidents.Design/methodology/approachDifferent types of properties in disaster plans have been identified and formalized using order‐sorted predicate logic, enabling automated comparison of plans and verification of such properties against logs by means of software tools. Actual disaster plans and logs have been used as a case study to show the working of the approach.FindingsThe automated approach can be used quite easily and result in important findings. For the case study disaster plans crucial differences were found that could have catastrophic consequences. Furthermore, it is shown that in the logs of a well‐known incident the disaster plan was not followed.Practical implicationsIf the approach is introduced in practice, disaster plans would be stored in a formal format, enabling the automated comparison of disaster plans, and immediate detection of derivation from a disaster plan in case of an incident.Originality/valueOther literature about the formal modelling of disaster plans that includes both structural and dynamical aspects and allows representation of organizational structure at multiple aggregation levels has not been found. Nor has comparing the disaster plans using such a formal model, and using the model of the disaster plan to check empirical traces for compliance with this plan, been addressed in prior literature.