Disaggregation Method Research Articles

Grain-Size Analysis of Ancient Deep-Marine Sediments Using Laser Diffraction

Grain-size analysis of siliciclastic sedimentary rocks provides critical information for interpreting flow dynamics and depositional environments in sedimentary systems and for analysing reservoir quality of sandstone. Methods such as sieving and thin-section analysis are time consuming and unsuited for large sample numbers. Laser diffraction particle analysis is quick and reliable for analysing 100s of samples, assuming successful disaggregation. Here, we evaluate this method utilizing samples from three siliciclastic formations in Northern Italy: the Miocene Castagnola and Marnoso-Arenacea Formations, and the Cretaceous to Palaeocene Gottero Formation, which vary in degree of lithification. We focus on: 1) methods of whole-rock disaggregation; 2) methods of subsampling sediment for laser diffraction analysis; and 3) comparison of thin-section analysis with laser-diffraction particle size analysis. Using an ultrasonic bath and a SELFRAG (high voltage selective fragmentation) as disaggregation tools, this study evaluates separation of whole, undamaged grains subsequently measured by laser diffraction analysis. We show that it is possible to disaggregate ancient, well cemented rocks using an ultrasonic bath. When disaggregating samples with the SELFRAG method, grain-size measurements become less accurate and less precise with increasing sample lithification and increased presence of cement. This is likely a combination of incomplete grain disaggregation in the SELFRAG and heterogeneity within samples. Following disaggregation, we compare sub-sampling methods using a stirrer plate versus a pipette. Both produce accurate analyses, but the stirrer method is the most reliable and replicable. A comparative small subsample method, run as one whole sample with no need for subdivision into aliquots, is found to be reliable and replicable but is more susceptible to heterogeneity within field samples. When comparing laser diffraction results to grain-size volume methods estimated from thin-section analysis, thin-section sand grains are overestimated, and clay/silt grains are inaccurate. These results provide a framework for understanding potential biases introduced through various sample preparation and measurement methods.

On the Emergence of Frequency Bias from Accumulating or Disaggregating Bias-Corrected Quantitative Precipitation Forecasts

Abstract Quantitative precipitation forecast (QPF) applications often demand accumulations of precipitation for both long- and short-duration time intervals. It is desired that the shorter-duration forecasts sum to the longer-duration accumulations spanning the same time period. In the context of calibration, it is further desired that both the subinterval and longer interval accumulations be similarly corrected to have near unit frequency bias on a spatial domain. This study examines two methods of achieving these goals for 6- and 24-h accumulation intervals: 1) the accumulation method bias corrects the 6-h forecasts and accumulates them to create the 24-h accumulations; and 2) the disaggregation method bias corrects the 24-h accumulation and then proportionately disaggregates the 24-h accumulation back into 6-h accumulations. The experiments for the study are done retrospectively so that a “perfect” bias correction is possible for each method. The results of the study show that neither method accomplishes the stated goal for the calibration because QPF placement and/or timing errors contribute to frequency bias in the course of accumulation or disaggregation. However, both methods can improve the frequency bias for both the subinterval and longer interval accumulations. The choice of method may hinge most strongly on the relative tolerance of bias for the subinterval accumulations versus the longer interval accumulation.

Dasymetry Dash Flood (DDF). A method for population mapping and flood exposure assessment in touristic cities

Population disaggregation methods are a land management tool that is necessary to robustly assess the exposure of populations to natural hazards. The aim of these methods is to translate population values from large spatial units to smaller spatial units. Due to their improvement, the accuracy in quantifying the population exposed to natural hazards has increased significantly in recent years. However, in the case of floods, where the actual exposure to the hazard depends on the height of the buildings, there is a methodological deficiency with regard to reaching the necessary level of detail. This is a methodological challenge that is exacerbated in urban areas specialising in tourism, where there are a large number of dwellings dedicated to the housing of tourists. In this paper we propose a 3D cartographic dasymetry (DDF) method that, based on cadastral information and the population and housing census, manages to solve these problems of flood hazard exposure assessment reasonably well. For validation, the results are compared with three widely used 2D methods. Our work shows that the proposed method offers better outputs for use in high-precision work; but also, when such detail is not necessary, more basic methods achieve results with only marginal differences.

A Novel Denoising Auto-Encoder-Based Approach for Non-Intrusive Residential Load Monitoring

Mounting concerns pertaining to energy efficiency have led to the research of load monitoring. By Non-Intrusive Load Monitoring (NILM), detailed information regarding the electric energy consumed by each appliance per day or per hour can be formed. The accuracy of the previous residential load monitoring approach relies heavily on the data acquisition frequency of the energy meters. It brings high overall cost issues, and furthermore, the differentiating algorithm becomes much more complicated. Based on this, we proposed a novel non-Intrusive residential load disaggregation method that only depends on the regular data acquisition speed of active power measurements. Additionally, this approach brings some novelties to the traditionally used denoising Auto-Encoder (dAE), i.e., the reconfiguration of the overlapping parts of the sliding windows. The median filter is used for the data processing of the overlapping window. Two datasets, i.e., the Reference Energy Disaggregation Dataset (REDD) and TraceBase, are used for test and validation. By numerical testing of the real residential data, it proves that the proposed method is superior to the traditional Factorial Hidden Markov Model (FHMM)-based approach. Furthermore, the proposed method can be used for energy data, disaggregation disregarding the brand and model of each appliance.

Joint Energy Disaggregation of Behind-the-Meter PV and Battery Storage: A Contextually Supervised Source Separation Approach

An increasing number of residential customers have installed hybrid rooftop solar battery systems (HRSBSs). Currently, most HRSBSs are installed behind-the-meter (BTM), where only customers’ net load is measured by smart meters. This invisibility poses significant challenges to the system operation. Disaggregating BTM PV generation and battery charging/discharging profile of customers can enhance the grid-edge observability. This article proposes an energy disaggregation method to jointly separate the PV generation and battery charging/discharging power from the net load. First, a Home Smart Battery Management model is built to simulate the battery charging/discharging profile. Second, an optimal disaggregation model is established based on a contextually supervised source separation method. Furthermore, the feature vectors of PV, load, and battery are extracted as the input of the disaggregation model. Case studies on two datasets from Australia and China show that the proposed method has a promising disaggregation performance.

A robust non-intrusive load disaggregation method with roof-top photovoltaics

As a promising technology dealing with electric behavior analysis and energy characteristic mining, non-intrusive load monitoring is popular with these people with energy conservation awareness. Coincidentally, these residents are with strong motivation to install roof-top photovoltaics. Therefore, it is a real challenge to implement non-intrusive load disaggregation considering PV integrations, which is rarely researched. In this paper, this problem is thoroughly investigated with an achievement of a corresponding robust load disaggregation approach. At the first stage, the problem formulation is established based on the evolved dictionary learning scheme, where the special features of PV, e.g., no switching pattern and continuous power variation, are addressed. Secondly, aiming at robustness of monitoring, the steady state disaggregation is enhanced by integrating with event-based detection, forming a hybrid validation strategy. Then, the whole problem is solved following the sparse coding principle. The proposed study is verified by both simulation tests and field measurements, and the results show that the proposed method is accurate and robust in non-intrusive load monitoring with PV consideration. Besides, it is compatible with various load signature utilization, leading to a practical disaggregation solution.

Forecasting the Emission of Carbon-di-oxide Equivalent in Key Sectors of India Using ARIMA Model

Climate Change is a global challenge and needs to be addressed immediately. The emission of Green House Gases in the atmosphere by anthropogenic factors is one of the major causes of Global Warming and Climate Change. India is working towards the control of global warming by focussing on controlling Green House Gas emissions. The emission of carbon-di-oxide in the atmosphere plays a predominant role in global warming. The Global Warming potential of the Green House Gases is measured in terms of carbon-dioxide equivalent CO2 (eq.). The annual CO2 (eq.) emission data from 2005-2015 from four key sectors viz., Energy, Industry Process and Product Use (IPPU), Agriculture Forestry and Land Use (AFOLU), and the Waste sector were considered in the study. Classical Temporal disaggregation methods Denton, Denton Cholette, Chow - Lin, Fernandez, and Litterman methods were employed to disaggregate the low frequency (annual) data to high frequency (quarterly) data. The analysis revealed that the Chow - Lin method of disaggregation best suited to disaggregate the CO2 (eq.) series for the three sectors except AFOLU with anAdjusted R square of 0.9 and the current Price GDP is the good indicator series for CO2 (eq.). The disaggregated data is modelled using ARIMA modelling. The CO2 (eq.) from 2021-Q1 to 2023-Q4 is forecasted using the fitted ARIMA model for each sector.

Dispersion simulations in an Andean city: Role of continuous traffic data in the spatio-temporal distribution of traffic emissions

Emissions information is essential for air quality modeling and management. However, detailed spatio-temporal emissions information is usually not available in urban centers of developing countries. In this study, we assess the ability of three spatio-temporal disaggregation methods for estimating traffic emissions and modeling dispersion of pollutants in the Aburrá Valley, a densely populated and complex terrain region in Colombia. The disaggregation methods, which differ on the detail in the input traffic information applied, are used to distribute annual traffic emissions for the whole study area to 1 km2 and hourly resolution. The distributed emissions are used as input in high-resolution dispersion simulations, and the simulated concentrations are compared against surface observations. In general, simulated pollution levels reproduce adequately observations. Results show that methods using a higher level of detail (continuous traffic information and road density) improved the representation of emissions dynamics, the dispersion of pollutants, and allocated higher emissions in zones of higher road density and traffic. Thus, these methods can be used to improve high-resolution simulations. Our results show that the selection of a disaggregation method should be rigorously analyzed in order to perform a spatio-temporal distribution of emissions. This approach provides insights for emissions control strategies and can be applied in other regions with available traffic information.

Bootstrap Prediction Intervals of Temporal Disaggregation

In this article, we propose an interval estimation method to trace an unknown disaggregate series within certain bandwidths. First, we consider two model-based disaggregation methods called the GLS disaggregation and the ARIMA disaggregation. Then, we develop iterative steps to construct AR-sieve bootstrap prediction intervals for model-based temporal disaggregation. As an illustration, we analyze the quarterly total balances of U.S. international trade in goods and services between the first quarter of 1992 and the fourth quarter of 2020.

An unsupervised data-driven approach for behind-the-meter photovoltaic power generation disaggregation

An increasing number of behind-the-meter (BtM) rooftop photovoltaic (PV) panels is being installed and maintained by site owners. However, invisible PV power generation (PVPG) will lead to the difficulty for system operators in power dispatch and affect the safety and stability of the power system. To better quantify BtM PVPG, a novel unsupervised data-driven disaggregation method freedom from PV system physical model assumption for BtM PVPG is proposed. After clustering the prosumers’ net load curves, a PVPG sensitivity estimation model is firstly built, based on the net load with approximate energy consumption (EC) and the corresponding irradiation data obtained from the pairing date. Then, an EC sensitivity model is developed according to the net load and temperature of the date with similar irradiation. Finally, a new net load disaggregation model is constructed by the PVPG sensitivity model with EC compensation. Case study based on Ausgrid data shows that the proposed method provides a better quality BtM PVPG disaggregation. The disaggregation accuracy improves by 5.06–5.87% as compared to the state-of-the-art methods.

Integrating Streambank Erosion with Overland and Ephemeral Gully Models Improves Stream Sediment Yield Simulation

HighlightsSimulation of overland (AnnAGNPS) and ephemeral gully (REGEM) erosion underestimated watershed sediment yield.Three methods were used to disaggregate contribution of streambank sediment loads to daily streamflow events.A rainfall-based method, with scaling factor and threshold term, had good performance and low bias in all watersheds.Calibrated model results indicated that most sediment yield was from streambank sources (78% to 93%).Abstract. Models used to guide watershed management must account for sediment from all erosion sources (interrill and rill, ephemeral gully (EG), and streambank), each of which operates at different spatial and temporal scales. Our hypothesis was that the use of separate models to explicitly simulate sediment yield contributions for each of these three sources would improve model agreement with measured watershed sediment yield data. We tested this hypothesis using AnnAGNPS (overland flow/erosion model), REGEM (EG erosion model), and field-measured streambank erosion disaggregated to an event basis using three methods in three watersheds (North Fork, Main Stem, and Irish Creek, Kansas). AnnAGNPS alone and in combination with REGEM underestimated sediment yields and had a poorly calibrated performance in all three watersheds, which reinforced the need to include sediment loads from stream sources. The three streambank sediment disaggregation methods each included two calibration terms: a scaling factor and a threshold term, which disaggregated 2-year-total streambank erosion measurements into event-based values based on rainfall, streamflow, or stream power. All three methods gave “very good” sediment-yield calibration results (event-based Ef = 0.6, PBIAS near 0%) for Main Stem and Irish Creek but gave “satisfactory” results only for the rainfall-based streambank disaggregation method in the evaluation watershed (North Fork). Calibrated model results indicated that most of the measured outlet sediment yields in the study watersheds were from streambank sources (78% to 93%), with less from overland erosion (7% to 22%) and little from EG erosion (0% to 7%). These methods could be equally effective in scaling and disaggregating stream-based sediment contributions without measured streambank erosion, but the calibration terms would lose physical meaning in reference to measured streambank values. The skill demonstrated by all three streambank disaggregation thresholds may provide a foundation for building simple, process-based streambank sediment yield simulation models. Keywords: Ephemeral gully erosion, Interrill and rill erosion, Modeling, Streambank erosion, Watershed.

Data Disaggregation for Inclusive Quality Education in Emergencies: The COVID-19 Experience in Ghana

The process of data analysis provides, undoubtedly, some of the major challenges facing organizations during the implementation of interventions in emergencies. The challenges are primarily due to the lack of direct access to beneficiaries and the rapidly evolving nature of emergencies. This paper outlines how Plan International's Making Ghanaian Girls Great! (MGCubed) project used phone-based surveys to assess the uptake of a Ghana Learning TV (GLTV) programme implemented in partnership with the government. Due to the emergency context and the need for real-time information to guide the implementation of this intervention, there was little time to undertake a major statistical analysis of survey data. This paper discusses how the MGCubed project adopted a simple data disaggregation method using a logic tree technique to gain valuable insights from the survey data. The method allowed for exploring the insights of the data set in real-time without requiring more complex and time-consuming analysis. All views expressed in this article are the author's and not of FCDO.

Overview of Non-Intrusive Load Monitoring: Probabilistic and Artificial Intelligence approaches

Reduction and conservation of electrical energy consumption in residential buildings is the main objective of Non-Intrusive Load Monitoring (NILM) techniques. NILM detects events and estimate the power consumption of individual appliances by analyzing the aggregate power consumption measured at the service entry. Indeed, our major contribution in this paper is to update research works on NILM methodologies by adding the most recent NILM methods proposed in the literature. In this paper we present an overview of NILM and energy disaggregation methods. Then, we discuss the challenges of this technique to provide useful recommendations for future research.

Simultaneous Estimation of Behind-the-Meter Solar and Wind Power at the Bulk Supply Point

The presence of distributed energy resources in private premises has observed a large deployment worldwide driven by favorable governmental policies and technology development. In Northern Ireland, the small-scale generation capacity including solar and wind power has fast-grown accounting for over 17% of peak load in 2021, which makes net demands more volatile and diminishes the accuracy of electric demand forecasting activities. Different from traditional load forecasting methods based on the relationship between heterogeneous variables and net demand fluctuation, a novel approach is proposed to separately forecast the invisible components (e.g. wind & solar power) within the net demand. The critical challenge in the separate forecasting is how to extract behind-the-meter components from the measured net demand data, namely net demand disaggregation, for which continuous wavelet transform is applied to bring out the detailed fluctuation of net demand and an iterative search algorithm is proposed to separate the components in the net demand. The results reported show that the proposed demand disaggregation method contributes to reduce the forecasting error by more than 14%. Methods like the proposed net demand disaggregation can benefit utilities and system operators in performing their daily activities regarding load forecasting in serving areas and power systems they operate.

Disaggregation of electricity and heating consumption in commercial buildings with building automation system data

Although understanding end uses in large commercial and institutional buildings is of great utility for achieving energy efficiency, most new and existing buildings still lack adequate submetering except for large end uses. End use disaggregation methods offer an alternative to fulfill weaknesses in a metering network, especially in large commercial and institutional buildings. Unlike non-intrusive load monitoring (NILM) methods that mostly rely on high frequency power consumption data and focus on residential buildings, this paper introduces an end-use disaggregation approach using building automation system (BAS) data. The BAS data provide information about the operational status of main energy-consuming systems and equipment such as fans, pumps, air handling unit (AHU) heating coils, and variable air volume (VAV) terminal reheat coils. The proposed method uses a suite of multiple linear regression models estimated by the genetic algorithm and a least-square solver to disaggregate heating and electricity use data using BAS data as predictors. The method was demonstrated with data from an academic office building in Ottawa, Canada. The accuracy was evaluated by comparing the disaggregation results to ground truth energy use data acquired from dedicated submeters. The findings suggest that the method can accurately disaggregate building-level hourly electricity and heating energy use into following end use categories, for electricity: (a) occupant-controlled loads (lighting and plug loads), (b) distribution loads (the electricity used by the AHUs); and for heating: (a) energy used by the AHU heating coils, (b) perimeter heating devices (reheat coils/and radiators).

Learning target-based preferences through additive models: An application in radiotherapy treatment planning

This article presents a new Multi-Criteria Decision Aiding preference disaggregation method based on an asymmetric target-based model. The decision maker's preferences are elicited considering the choices made given a set of comparisons among pairs of solutions (the stimuli). It is assumed that the decision maker has a reference value (target) for the stimulus. Solutions that do not comply with this reference value for some of the criteria dimensions considered will be penalized, and an inferred weight is associated with each dimension to calculate a penalty score for each solution. One of the differentiating features of the proposed model when compared with other existing models is the fact that only solutions that do not meet the target are penalized. The target is not interpreted as an ideal solution, but as a set of threshold values that should be taken into account when choosing a solution. The proposed approach was applied to the problem of choosing radiotherapy treatment plans, using a set of retrospective cancer cases treated at the Portuguese Oncology Institute of Coimbra. Using paired comparison choices made by one radiation oncologist, the preference model was built and was tested with in-sample and out-of-sample data. It is possible to conclude that the preference model is capable of representing the radiation oncologist's preferences, presenting small mean errors and leading, most of the time, to the same treatment plan chosen by the radiation oncologist.

Power disaggregation in commercial buildings considering unmonitored facilities and multiple routines

Estimating energy conservation for commercial buildings is challenging due to unreliable power predictions. Previously proposed disaggregation technologies require site surveys to decompose a single meter into multiple elements and address commercial buildings’ unknown power consumption/equipment. We propose a novel energy disaggregation method for the energy consumption of commercial buildings with limited facility status monitoring points. The method accurately estimates and identifies the energy consumption of each monitored/unmonitored facility using the statuses of the monitored facilities. The unmonitored facility consumption is evaluated using the linear regression residual of the monitored facilities and is clustered by daily routine. Daily variations in the magnitude of the power consumption of unmonitored facilities are estimated by assigning a linear basis function model with different diurnal periodicities to each cluster. The monitored facility regression parameters and basis functions are determined according to the total energy consumption of the building and the operational statuses of the monitored facilities. When applied to a commercial building, our method estimated the power consumption of each facility more accurately than the conventional method, without accessing the building’s operational calendar or the equipment power rating. Thus, the proposed method can be used to help energy managers make informed decisions regarding commercial buildings.

Energy disaggregation using variational autoencoders

Non-intrusive load monitoring (NILM) is a technique that uses a single sensor to measure the total power consumption of a building. Using an energy disaggregation method, the consumption of individual appliances can be estimated from the aggregate measurement. Recent disaggregation algorithms have significantly improved the performance of NILM systems. However, the generalization capability of these methods to different houses as well as the disaggregation of multi-state appliances are still major challenges. In this paper we address these issues and propose an energy disaggregation approach based on the variational autoencoders framework. The probabilistic encoder makes this approach an efficient model for encoding information relevant to the reconstruction of the target appliance consumption. In particular, the proposed model accurately generates more complex load profiles, thus improving the power signal reconstruction of multi-state appliances. Moreover, its regularized latent space improves the generalization capabilities of the model across different houses. The proposed model is compared to state-of-the-art NILM approaches on the UK-DALE and REFIT datasets, and yields competitive results. The mean absolute error reduces by 18% on average across all appliances compared to the state-of-the-art. The F1-Score increases by more than 11%, showing improvements for the detection of the target appliance in the aggregate measurement.

Geospatial Data Disaggregation through Self-Trained Encoder–Decoder Convolutional Models

Datasets collecting demographic and socio-economic statistics are widely available. Still, the data are often only released for highly aggregated geospatial areas, which can mask important local hotspots. When conducting spatial analysis, one often needs to disaggregate the source data, transforming the statistics reported for a set of source zones into values for a set of target zones, with a different geometry and a higher spatial resolution. This article reports on a novel dasymetric disaggregation method that uses encoder–decoder convolutional neural networks, similar to those adopted in image segmentation tasks, to combine different types of ancillary data. Model training constitutes a particular challenge. This is due to the fact that disaggregation tasks are ill-posed and do not entail the direct use of supervision signals in the form of training instances mapping low-resolution to high-resolution counts. We propose to address this problem through self-training. Our method iteratively refines initial estimates produced by disaggregation heuristics and training models with the estimates from previous iterations together with relevant regularization strategies. We conducted experiments related to the disaggregation of different variables collected for Continental Portugal into a raster grid with a resolution of 200 m. Results show that the proposed approach outperforms common alternative methods, including approaches that use other types of regression models to infer the dasymetric weights.

Uncertainty of Intensity-Duration-Frequency Curves Due to Adoption or Otherwise of the Temperature Climate Variable in Rainfall Disaggregation

Most areas around the world lack fine rainfall records which are needed to derive Intensity-Duration-Frequency (IDF) curves, and those that are available are in the form of daily data. Thus, the disaggregation of rainfall data from coarse to fine temporal resolution may offer a solution to that problem. Most of the previous studies have adopted only historical rainfall data as the predictor to disaggregate daily rainfall data to hourly resolution, while only a few studies have adopted other historical climate variables besides rainfall for such a purpose. Therefore, this study adopts and assesses the performance of two methods of rainfall disaggregation one uses for historical temperature and rainfall variables while the other uses only historical rainfall data for disaggregation. The two methods are applied to disaggregate the current observed and projected modeled daily rainfall data to an hourly scale for a small urban area in the United Kingdom. Then, the IDF curves for the current and future climates are derived for each case of disaggregation and compared. After which, the uncertainties associated with the difference between the two cases are assessed. The constructed IDF curves (for the two cases of disaggregation) agree in the sense that they both show that there is a big difference between the current and future climates for all durations and frequencies. However, the uncertainty related to the difference between the results of the constructed IDF curves (for the two cases of disaggregation) for each climate is considerable, especially for short durations and long return periods. In addition, the projected and current rainfall values based on disaggregation case which adopts historical temperature and rainfall variables were higher than the corresponding projections and current values based on only rainfall data for the disaggregation.