Level Of Data Aggregation Research Articles

The information layer: shaping data into meaning with interactive tools

Abstract Valorization and interpretation of health data are contingent upon a robust analytical framework. This framework encompasses essential processes such as data cleaning, validation, and the critical evaluation of bias and missing data. A nuanced understanding of these factors is crucial for contextualizing data through stratification, developing comprehensive analysis plans, and addressing interactions, covariates, and confounders. Such meticulous methodologies guard against ‘data fishing expeditions’ that could otherwise lead to erroneous conclusions. Healthybelgium.be exemplifies our approach for deciding on appropriate levels of data aggregation and stratification. This study serves as a foundation for deriving conclusions and formulating hypotheses regarding the health status of the national and various sub-populations. Our discussion will cover the limitations of the questions that can be answered with the available data, the resulting opportunities for data enhancement, and the subsequent valorization of findings for knowledge advancement. Furthermore, we will present the interactive tools (through the use of R Shiny) developed by Sciensano to disseminate information broadly, making it accessible and comprehensible to diverse audiences. These tools embody a balance of strength and limitation, facilitating an informed dialogue among stakeholders. The ongoing refinement of these tools, influenced by continuous learning and insights, exemplifies our commitment to evolving public health intelligence.

The Effect of Increasing Aggregation Levels of Electrical Consumption Data on Renewable Energy Community (REC) Analyses

The growing number of renewable energy communities (RECs) exemplifies the potential of citizen-driven actions towards a more sustainable future. However, obtaining hourly measured consumption data from REC members remains challenging, hindering accurate feasibility studies for the development of communities. This study examines the impact of estimating hourly consumption from aggregated data on REC analysis results. A case study with real consumption data from diverse users, representative of a typical community in Tuscany, Italy, was analysed to investigate various influencing factors. Multiple scenarios were simulated: two open-source tools estimated energy production from the community’s PV plants, and two REC configurations were considered—one with consumers and prosumers and another with consumers and a producer (with the same total installed power). Additionally, three locations were evaluated to consider the importance of geographical positioning. The study revealed that the impact of consumption data aggregation is more significant in scenarios with low energy sharing, such as the scenario where prosumers were replaced with a producer. Geographical positioning showed no major discrepancies in energy and economic outcomes, implying that using estimated hourly consumption data from aggregated data has a relevant impact regardless of location. Furthermore, different weather files did not affect the impact of aggregated consumption data.

The use of the Bennet indicators and their transitive versions for scanner data analysis

Although modern price index theory is based on comparisons of ratios of prices, quantities and expenditures, we may be more interested in the magnitude of differences in these characteristics in many business applications. The benefit of using these differences is that there is no problem associated with the occurrence of zero prices and quantities, a problem that arises when we work with ratios. In practice, we most often care about decomposing value difference into indicators of contributions from price and quantity differences. The best-known price and quantity indicators are the Bennet indicators, which are not transitive. Although there have been papers in the literature that propose a transitive version of the Bennet indicators, they deal with comparisons across firms in cross-section or panel contexts. This paper revises the price and quantity Bennet indicators and their multilateral versions for the analysis of scanner data. Specifically, i nstead o f c onsidering c omparisons across firms, c ountries or r egions, the t ransitive versions of t he Bennet i ndicators a re a dapted to work on scanner data sets observed over a fixed time w indow. Since the scanner data sets have a high turnover of products, which can make it difficult to interpret the difference in sales values over the compared time periods, the paper also considers a matched sample approach. One of the objectives of the study is to compare bilateral and multilateral Bennet indicator results across all available products or strictly matched products over time. It also examines the impact of data filters used and the level of data aggregation on the price and quantity Bennet indicators. According to the best author’s knowledge, this study is a pioneer in the field of implementing the multilateral Bennet indicators in scanner data analysis.

The influence of data aggregation levels on accident severity researches

Abstract This paper aims at assessing the influence of data aggregation on the estimation of the accidents’ severity. For this purpose, 27,400 accidents are investigated on three aggregation levels: individually (idl) which is highly disaggregated hourly (hrl), and daily (dil). A Severity Score (SS) is defined based on the injuries’ severity and relative cost weights. Multiple Linear Regressions are developed by assessing the SS’s variation as a collective explanatory effect of 83 predictors related to road environment, weather and lighting conditions, adherence conditions, and number of the involved vehicles. F and t tests are employed in order to assess the developed models and to investigate which variable is of statistical significance. This research shows that the higher the level of aggregation is, the higher the coefficient of determination. SS_dil model has the higher R2 value and direct relationships between SS and the number of accidents occurring in “U” curves, on roads with no shoulders, where the roadway was covered with slime or black ice and where the roadway was wet was found. One of the interesting findings of this paper shows that in the models with low R2 values (based on individual and hourly aggregated data), snow and fog significantly explain the variation of SS. As these are seasonally and short time phenomena, a further research should be focused on how the estimated models based on individual or hourly aggregated data could be improved by adding other significant predictors related to driver’s and vehicle’s characteristics and road environment’ features.

The relationships between Urban Tree Canopy Cover and Crime in São Paulo City, Brazil

Prior research has indicated lower crime rates in areas with greater tree canopy cover predominantly in the Global North. There are few studies from the Global South, and more specifically in Latin America. Given the high prevalence of crime in Latin American cities, the need to address social and environmental inequalities using nature-based solutions is urgent. This study examines crime and tree canopy cover through a series of spatial analyses for São Paulo city, Brazil, while controlling for potential confounders. We tested six levels of data aggregation that were combined with three different crime types and two denominators of crime, to account for spurious findings from Modifiable Areal Unit Problems (MAUP). Of the 36 models, a majority of 27 models (75 %) indicate a negative relationship between crime and tree cover canopy, while 4 models (11 %) show a positive correlation, and 5 models (14 %) suggest no statistically significant relationship. Our best models allow us to infer that 10 % greater tree canopy cover is associated with a 1.20 % fewer property, personal and total reported crimes, with statistical significance at the p < 0.001 level. These findings support our hypothesis that there is a negative relationship between tree canopy cover and crime, with a stronger association observed for property crime than personal or total crime. The negative relationship persisted independently of the level of data aggregation, crime types, crime denominator and spatial models specifications (lag and SARMA), avoiding statistical bias of MAUP. The results follow the literature that have observed tree cover associated with lower crime rates, for different types of crime, while adding a new climatic and cultural context to the evidence base.

Evaluating the robustness of parameter estimates in cognitive models: A meta-analytic review of multinomial processing tree models across the multiverse of estimation methods.

Researchers have become increasingly aware that data-analysis decisions affect results. Here, we examine this issue systematically for multinomial processing tree (MPT) models, a popular class of cognitive models for categorical data. Specifically, we examine the robustness of MPT model parameter estimates that arise from two important decisions: the level of data aggregation (complete-pooling, no-pooling, or partial-pooling) and the statistical framework (frequentist or Bayesian). These decisions span a multiverse of estimation methods. We synthesized the data from 13,956 participants (164 published data sets) with a meta-analytic strategy and analyzed the magnitude of divergence between estimation methods for the parameters of nine popular MPT models in psychology (e.g., process-dissociation, source monitoring). We further examined moderators as potential sources of divergence. We found that the absolute divergence between estimation methods was small on average (<.04; with MPT parameters ranging between 0 and 1); in some cases, however, divergence amounted to nearly the maximum possible range (.97). Divergence was partly explained by few moderators (e.g., the specific MPT model parameter, uncertainty in parameter estimation), but not by other plausible candidate moderators (e.g., parameter trade-offs, parameter correlations) or their interactions. Partial-pooling methods showed the smallest divergence within and across levels of pooling and thus seem to be an appropriate default method. Using MPT models as an example, we show how transparency and robustness can be increased in the field of cognitive modeling. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Desiderata for discoverability and FAIR adoption of health data hubs

BackgroundThe future European Health Research and Innovation Cloud (HRIC), as fundamental part of the European Health Data Space (EHDS), will promote the secondary use of data and the capabilities to push the boundaries of health research within an ethical and legally compliant framework that reinforces the trust of patients and citizens. ObjectiveThis study aimed to analyse health data management mechanisms in Europe to determine their alignment with FAIR principles and data discovery generating best.practices for new data hubs joining the HRIC ecosystem. In this line, the compliance of health data hubs with FAIR principles and data discovery were assessed, and a set of best practices for health data hubs was concluded. MethodsA survey was conducted in January 2022, involving 99 representative health data hubs from multiple countries, and 42 responses were obtained in June 2022. Stratification methods were employed to cover different levels of granularity. The survey data was analysed to assess compliance with FAIR and data discovery principles. The study started with a general analysis of survey responses, followed by the creation of specific profiles based on three categories: organization type, function, and level of data aggregation. ResultsThe study produced specific best practices for data hubs regarding the adoption of FAIR principles and data discoverability. It also provided an overview of the survey study and specific profiles derived from category analysis, considering different types of data hubs. ConclusionsThe study concluded that a significant number of health data hubs in Europe did not fully comply with FAIR and data discovery principles. However, the study identified specific best practices that can guide new data hubs in adhering to these principles. The study highlighted the importance of aligning health data management mechanisms with FAIR principles to enhance interoperability and reusability in the future HRIC.

Robust Demand Estimation With Customer Choice-Based Models for Sales Transaction Data

We develop a novel statistical method to estimate customer choice among a firm’s portfolio of offerings when the firm cannot directly observe customers who choose not to purchase any product. This censored demand problem is prevalent in many industries such as hotels, airlines, and retail. Although several methods have been proposed to address this problem, they require some level of data aggregation across arrivals and/or choice sets, which results in information loss and potentially biased estimates. Therefore, they have limited applicability in an environment where the prices of a firm’s portfolio of offerings vary over time and sometimes even across different customers. Our proposed method combines several desirable properties, which makes it a better fit for realistic datasets where the available choice sets or attributes of the products in the choice sets change over time. We consider two additional types of information for identification of our model parameters: (1) additional mild assumptions on the customers’ utility function, and (2) external information about a firm’s market share. We then develop a robust estimation procedure that accounts for inaccuracies in either information type and let the data determine the best approach. Through Monte Carlo simulations, we show that our approach provides promising predictions of customer choice behavior when compared with other generally used methods and clearly outperforms those methods in scenarios where the product prices change frequently over time. Utilizing a real hotel transaction dataset provided by Oracle Labs, we further illustrate the improved estimation accuracy of our method compared to benchmark methods. Relative to existing approaches for estimating customer choice-based models, our proposed methodology better suits environments employing dynamic pricing and personalized offering practices, such as hospitality or online retailing.

Estimating the elasticity of substitution when compiling the CES cost of living index on scanner data

AbstractScanner data are electronic transaction data that specify turnover and the number of items sold by barcodes, e.g., the Global Trade Article Number. These data are of particular value and interest to theorists and practitioners who wish to measure the Cost of Living Index or the Consumer Price Index, since their complete content makes it possible to compute any price index formula, including superlative indices or CES (Constant Elasticity Substitution) indices. Since the CES index requires the estimation of the elasticity of substitution, this paper focuses on verifying various methods of estimating this parameter based on scanner data. The paper considers both algebraic methods and methods based on the panel regression approach. The main achievement of the paper is the separation of the main factors that affect the estimated value of the elasticity of substitution, i.e., the type of data filter used and the level of data aggregation. The paper also verifies how the elasticity of substitution estimates affect the differences between the values of the CES indices based on these estimates.

Considering the development of organic farming in Poland at different territorial aggregation levels

The aim of this paper is to show and assess how the level of data aggregation can change the way the development of organic agriculture in Poland in 2014–2021 is perceived. A composite indicator of eco-farming at different levels of territorial aggregation and in different time horizons has been built taking into account sub-components and individual indicators. The performance of organic agriculture with the initial effects of the pandemic (for 2020–2021) was also quantified. The results showed that, in populationally larger territorial units (NUTS-1, NUTS-2, NUTS-3), organic farming characteristics are less variable, whereas the same characteristics show greater variability in smaller territorial units (LAU-1, LAU-2). This reflects the strong regional character of eco-agriculture. It can be observed especially in northern and south-eastern Poland. Moreover, although the highest performance of organic farming was observed in 2021, the effects of the pandemic crisis were highly heterogeneous by location. The results may be crucial for farmers and policymakers in planning sustainable agricultural strategies and building resilient organic regions.

Accounting for sample overlap in economics meta‐analyses: The generalized‐weights method in practice

AbstractMeta‐analyses in economics frequently exhibit considerable overlap among primary samples. If not addressed, sample overlap leads to efficiency losses and inflated rates of false positives at the meta‐analytical level. In previous work, we proposed a generalized‐weights (GW) approach to handle sample overlap. This approach effectively approximates the correlation structure between primary estimates using information on sample sizes and overlap degrees in the primary studies. This paper demonstrates the application of the GW method to economics meta‐analyses, addressing practical challenges that are likely to be encountered. We account for variations in data aggregation levels, estimation methods, and effect size metrics, among other issues. We derive explicit covariance formulas for different scenarios, evaluate the accuracy of the approximations, and employ Monte Carlo simulations to demonstrate how the method enhances efficiency and restores the false positive rate to its nominal level.

Secure data aggregation using quantum key management in IoT networks

The Internet of Things and Quantum Computing raise concerns, as Quantum IoT defines security that exploits quantum security management in IoT. The security of IoT is a significant concern for ensuring secure communications that must be appropriately protected to address key distribution challenges and ensure high security during data transmission. Therefore, in the critical context of IoT environments, secure data aggregation can provide access privileges for accessing network services. "Most data aggregation schemes achieve high computational efficiency; however, the cryptography mechanism faces challenges in finding a solution for the expected security desecration, especially with the advent of quantum computers utilizing public-key cryptosystems despite these limitations. In this paper, the Secure Data Aggregation using Quantum Key Management scheme, named SDA-QKM, employs public-key encryption to enhance the security level of data aggregation. The proposed system introduces traceability and stability checks for the keys to detect adversaries during the data aggregation process, providing efficient security and reducing authentication costs. Here the performance has been evaluated by comparing it with existing competing schemes in terms of data aggregation. The results demonstrate that SDA-QKM offers a robust security analysis against various threats, protecting privacy, authentication, and computation efficiency at a lower computational cost and communication overhead than existing systems.

AI -Driven Data Aggregation Level Smart Contracts for Blockchain Healthcare Insurance Claims Adjudication

The current model of healthcare insurance claim adjudication faces significant challenges, in particular fraudulent claims which significantly strain the economy. This paper introduces a novel approach that integrates Data Aggregation, Artificial Intelligence (AI), and Blockchain technologies to enhance the efficiency of healthcare claim adjudication. We present in this study an initial framework that integrates an AI-enhanced smart contract at the data aggregation level. A new concept that enhances the existing smart contract models commonly discussed in blockchain-related insurance literature. Our primary goal is to improve fraud detection and maximize the use of extensive data generated within the insurance ecosystem for prediction, learning, and strategic planning. The concept of an AI-data aggregation-level smart contract, introduced here, holds promise for broad application across various blockchain platforms, not limited to our discussion on healthcare claims insurance adjudication.

An integrated individual-level trait-based phytoplankton dataset from transitional waters

Functional trait-based approaches have undergone an extraordinary expansion in phytoplankton ecology. Morpho-functional traits have been shown to vary both within and between populations and species, potentially affecting individual fitness and the network of inter-individual relationships. Here we integrate six fully harmonized phytoplankton morpho-functional trait datasets, characterized by a fine data grain, reporting individual-level data over a large biogeographical area. Datasets refer to transitional water ecosystems, from five biogeographical areas: Northern Atlantic Ocean (Scotland), South-Western Atlantic Ocean (Brazil), South-Western Pacific Ocean (Australia), Indo Pacific Ocean (Maldives) and Mediterranean Sea (Greece and Turkey). The integrated dataset includes 127311 individual phytoplankton records with sampling locations, taxonomic and morphometric information according to Darwin Core standards and semantic annotations. The six FAIR datasets are openly available in the LifeWatch Italy data portal. The datasets have already been used for morpho-functional analyses and hypothesis testing on phytoplankton guilds at different levels of data aggregation and scale, from local to global.

Measuring Inequality Using Geospatial Data

Abstract The main challenge in studying inequality is limited data availability, which is particularly problematic in developing countries. This study constructs a measure of light-based geospatial income inequality (LGII) for 234 countries/territories from 1992 to 2013 using satellite data on night-lights and gridded population data. Key methodological innovations include the use of varying levels of data aggregation, and a calibration of the lights–prosperity relationship to match traditional inequality measures based on income data. The new LGII measure is significantly correlated with cross-country variation in income inequality. Within countries, the light-based inequality measure is also correlated with measures of energy efficiency and the quality of population data. Two applications of the data are provided in the fields of health economics and international finance. The results show that light- and income-based inequality measures lead to similar results, but the geospatial data offer a significant expansion of the number of observations.

Agri-food importing firms amid a global health crisis

This paper exploits daily customs transaction data on the universe of Swiss agri-food importing firms to assess the response of firms to a global shock. Estimating a linear model that regresses product-level import margins on daily COVID19 shocks and a host of fixed effects, we find that the pandemic had a substantial trade-reducing effect on imports. The trade effects were driven mainly by a reduction in the number of importing firms (i.e., 63% of the total effect), and much less by the number of products imported and the average import value per product per firm. We explore several sources of heterogeneity and show, among others, that larger and incumbent firms were affected more by the trade adjustments. Our results also reveal that the relative contribution of each import margin to the decline in aggregate imports depends on the level of data aggregation (i.e., daily, weekly or monthly). Finally, we validate and confirm our main findings by testing two mechanisms: (i) third-country supply-side effects using insights from structural gravity models and (ii) changes to consumer demand using consumer mobility, and retailer and consumer scanner data.

Measuring welfare with the use of the MIMIC approach: evidence for Poland

The growing interest in the concept of using welfare categories as a measure of economic or social development results from the fact that dominant income categories have been replaced by solutions related to individuals’ basic, existential and higher-order needs being fulfilled. The transition from analysing the poverty rate category, through the various approaches to material deprivation, to the perception of welfare in economic, subjective, or hidden terms is visible. The main purpose of the study is to estimate the subjective level of welfare of households in Poland. The subjective level of various socio-economic phenomena is often difficult to estimate; therefore, the study uses the approach of structural modelling for the multiple indicators and multiple causes model (MIMIC), which assumes the presence of a latent variable. The research is based on data from the Social Diagnosis panel study for household level in Poland. Based on the construction and positive substantive and statistical verification of the model the results show that subjectively the best welfare situation was identified mainly for households located in countryside areas, where even the lowest estimated levels of welfare exceed the highest estimated levels for cities of various sizes. Investigating the spatial distribution, the highest levels of subjective welfare were recorded for the Lubelskie and Swietokrzyskie voivodeships, and thanks to moving to a higher spatial data aggregation level (to the sub-regional), a more detailed assessment of spatial units was possible. In further research, individualised voivodeship models will be estimated to capture a more accurate differentiation of the influence of MIMIC model variables. A similar direction of analysis is anticipated for sub-regional data.

Quality adjusted GEKS-type indices for price comparisons based on scanner data

A wide variety of retailers (supermarkets, home electronics, Internet shops, etc.) provide scanner data containing information at the level of the barcode, e.g. the Global Trade Item Number (GTIN). As scanner data provide complete transaction information, we may use the expenditure shares of items as weightsfor calculating price indices at the lowest (elementary) level of data aggregation. The challenge here is the choice of the index formula which should be able to reduce chain drift bias and substitution bias. Multilateral index methods seem to be the best choice due to the dynamic character of scanner data. These indices work on a wholetime window and are transitive, which is key to the elimination of the chain drift effect. Following what is called an identity test, however, it may be expected that even when only prices return to their original values, the index becomes one. Unfortunately, the commonly used multilateral indices (GEKS, CCDI, GK, TPD, TDH) do not meet the identity test. The paper discusses the proposal of two multilateral indices and their weighted versions. On the one hand, the design of the proposed indices is based on the idea of the GEKS index. On the other hand, similarly to the Geary-Khamis method, it requires quality adjusting. It is shown that the proposed indices meet the identity test and most other tests. In an empirical and simulation study, these indices are compared with the SPQ index, which is relatively new and also meets the identity test. The analytical considerations as well as empirical studies confirm the high usefulness of the proposed indices.

Income‐poor, asset‐rich? The role of homeownership in shaping the welfare position of the elderly

AbstractThe study explores the distribution of elderly households in terms of combinations of different levels of income and wealth and relates to the typically greater homeownership rates in this group than in the younger generation. Using microdata from the Luxembourg Wealth Study for 12 countries it demonstrates that ‘income‐poor, asset‐rich’ elderly households are a quite marginal category. However, the identification of this group is crucial from the policy perspective, as it constitutes a target for actions aiming at increasing the ability of the elderly households to use housing wealth in such a way as to improve their financial well‐being. The results also suggest that higher homeownership rates among elderly households, as compared to non‐elderly households, are not accompanied by proportionally greater household wealth. Nevertheless, when the level of data aggregation is reduced, there is some evidence that homeownership is positively associated with the welfare position of the elderly.

Check the box! How to deal with automation bias in AI-based personnel selection.

Artificial Intelligence (AI) as decision support for personnel preselection, e.g., in the form of a dashboard, promises a more effective and fairer selection process. However, AI-based decision support systems might prompt decision makers to thoughtlessly accept the system's recommendation. As this so-called automation bias contradicts ethical and legal requirements of human oversight for the use of AI-based recommendations in personnel preselection, the present study investigates strategies to reduce automation bias and increase decision quality. Based on the Elaboration Likelihood Model, we assume that instructing decision makers about the possibility of system errors and their responsibility for the decision, as well as providing an appropriate level of data aggregation should encourage decision makers to process information systematically instead of heuristically. We conducted a 3 (general information, information about system errors, information about responsibility) x 2 (low vs. high aggregated data) experiment to investigate which strategy can reduce automation bias and enhance decision quality. We found that less automation bias in terms of higher scores on verification intensity indicators correlated with higher objective decision quality, i.e., more suitable applicants selected. Decision makers who received information about system errors scored higher on verification intensity indicators and rated subjective decision quality higher, but decision makers who were informed about their responsibility, unexpectedly, did not. Regarding aggregation level of data, decision makers of the highly aggregated data group spent less time on the level of the dashboard where highly aggregated data were presented. Our results show that it is important to inform decision makers who interact with AI-based decision-support systems about potential system errors and provide them with less aggregated data to reduce automation bias and enhance decision quality.