Disaggregation Techniques Research Articles

Deep recurrent extreme learning machine for behind-the-meter photovoltaic disaggregation

In recent years, sustainable sources of energies attract significant interest due to the serious environmental issues of fossil fuels. Rooftop photovoltaic (PV) panels are among the important power generation technologies exploited in many modern countries. Most PV panels are installed behind-the-meter (BTM), resulting in a lack of observability of actual load and PV power generation in a power distribution system. This paper proposes a novel supervised spatiotemporal approach to accurately disaggregate the net-load data of a set of neighboring residential units. To this end, spatiotemporal correlations of a group of neighboring residential units are modeled using a weighted undirected graph where the nodes store the temporal features. The edges reflect the spatial correlation between neighboring residential units and are determined by the information analysis technique. Afterward, a generative graph attention recurrent neural network (RNN) is devised for capturing highly nonlinear patterns of input graphs using an RNN encoder. In addition, to boost the generalization capacity and robustness of the proposed model, we reconstruct the input graphs using sparse contractive decoders. Finally, an extreme learning machine (ELM) neural network is employed to disaggregate the input net-load time-series of a set of residential units using the extracted complex spatiotemporal patterns. Experimental results on the real-world Pecan Street dataset demonstrate the superiority of the proposed RNN-ELM method over the recent BTM disaggregation techniques.

Harvesting and Disaggregation: An Overlooked Step in Biofilm Methods Research.

Biofilm methods consist of four distinct steps: growing the biofilm in a relevant model, treating the mature biofilm, harvesting the biofilm from the surface and disaggregating the clumps, and analyzing the sample. Of the four steps, harvesting and disaggregation are the least studied but nonetheless critical when considering the potential for test bias. This article demonstrates commonly used harvesting and disaggregation techniques for biofilm grown on three different surfaces. The three biofilm harvesting and disaggregation techniques, gleaned from an extensive literature review, include vortexing and sonication, scraping and homogenization, and scraping, vortexing and sonication. Two surface types are considered: hard non-porous (polycarbonate and borosilicate glass) and porous (silicone). Additionally, we provide recommendations for the minimum information that should be included when reporting the harvesting technique followed and an accompanying method to check for bias.

Learning from Non-experts: An Interactive and Adaptive Learning Approach for Appliance Recognition in Smart Homes

With the acceleration of Information and Communication Technologies and the Internet-of-Things paradigm, smart residential environments , also known as smart homes , are becoming increasingly common. These environments have significant potential for the development of intelligent energy management systems and have therefore attracted significant attention from both academia and industry. An enabling building block for these systems is the ability of obtaining energy consumption at the appliance-level. This information is usually inferred from electric signals data (e.g., current) collected by a smart meter or a smart outlet, a problem known as appliance recognition . Several previous approaches for appliance recognition have proposed load disaggregation techniques for smart meter data. However, these approaches are often very inaccurate for low consumption and multi-state appliances. Recently, Machine Learning (ML) techniques have been proposed for appliance recognition. These approaches are mainly based on passive MLs, thus requiring pre-labeled data to be trained. This makes such approaches unable to rapidly adapt to the constantly changing availability and heterogeneity of appliances on the market. In a home setting scenario, it is natural to consider the involvement of users in the labeling process, as appliances’ electric signatures are collected. This type of learning falls into the category of Stream-based Active Learning (SAL). SAL has been mainly investigated assuming the presence of an expert , always available and willing to label the collected samples. Nevertheless, a home user may lack such availability, and in general present a more erratic and user-dependent behavior. In this article, we develop a SAL algorithm, called K -Active-Neighbors (KAN), for the problem of household appliance recognition. Differently from previous approaches, KAN jointly learns the user behavior and the appliance signatures. KAN dynamically adjusts the querying strategy to increase accuracy by considering the user availability as well as the quality of the collected signatures. Such quality is defined as a combination of informativeness , representativeness , and confidence score of the signature compared to the current knowledge. To test KAN versus state-of-the-art approaches, we use real appliance data collected by a low-cost Arduino-based smart outlet as well as the ECO smart home dataset. Furthermore, we use a real dataset to model user behavior. Results show that KAN is able to achieve high accuracy with minimal data, i.e., signatures of short length and collected at low frequency.

The p-Lagrangian relaxation for separable nonconvex MIQCQP problems

This paper presents a novel technique to compute Lagrangian bounds for nonconvex mixed-integer quadratically constrained quadratic programming problems presenting a separable structure (i.e., a separable problems) such as those arising in deterministic equivalent representations of two-stage stochastic programming problems. In general, the nonconvex nature of these models still poses a challenge to the available solvers, which do not consistently perform well for larger-scale instances. Therefore, we propose an appealing alternative algorithm that allows for overcoming computational performance issues. Our novel technique, named the p-Lagrangian decomposition, is a decomposition method that combines Lagrangian decomposition with mixed-integer programming-based relaxations. These relaxations are obtained using the reformulated normalised multiparametric disaggregation technique and can be made arbitrarily precise by means of a precision parameter p. We provide a technical analysis showing the convergent behaviour of the approach as the approximation is made increasingly precise. We observe that the proposed method presents significant reductions in computational time when compared with a previously proposed techniques in the literature and the direct employment of a commercial solver. Moreover, our computational experiments show that the employment of a simple heuristic can recover solutions with small duality gaps.

A piecewise relaxation for quadratically constrained problems based on a mixed-radix numeral system

Non-convex quadratically constrained problems frequently appear in chemical engineering when optimizing process networks. Some of these problems can be solved to global optimality by deterministic solvers like BARON and ANTIGONE that mostly use linear programming relaxations coupled with spatial branch and bound. An alternative is to rely on piecewise relaxations, which work by simultaneously partitioning the domain of one variable in every bilinear term and can be significantly tighter, even when setting the number of intervals in the partitions, N, to a small value. In this short note, we generalize the mixed-integer linear programming relaxation formulation from the multiparametric disaggregation technique, to benefit from a logarithmic partitioning scheme in a wide variety of settings. The idea is to select the optimal interval in the partition of a variable by using a mixed-radix numeral system, following the prime factorization of N.

Value-Driven Multiple Criteria Sorting with Probabilistic Linguistic Information Considering Uncertain Assignment Examples

Multiple criteria sorting (MCS) dedicates to assigning alternatives to one of the predefined ordered categories according to their evaluation information on multiple criteria. The utility (value) function-based sorting is a popular MCS procedure, which requires decision-makers to express their preferences through assignment examples. By taking the assignment examples as reference alternatives, the additive value function, as the preferred model of a decision maker, can be built using the preference disaggregation technique. However, the existing literature hardly considered people’s hesitancy when determining assignment examples, and ignored applying linguistic evaluation information on qualitative criteria. To fill these research gaps, this study proposes a value-driven MCS procedure with probabilistic linguistic information considering uncertain assignment examples. Specifically, the probability linguistic term set, as a flexible information representation tool, is introduced to express the hesitancy of decision-makers regarding assignment examples and the performance of alternatives on qualitative criteria. Besides, to comprehensively reflect the preference of a decision-maker, a weighted additive value function is proposed based on the preference disaggregation technique to calculate the comprehensive scores of alternatives in which the weights are determined by the best-worst method. Finally, a case study on the sorting of down coats for sale demonstrates the applicability and superiority of our proposed method.

Solving data envelopment analysis models with sum-of-fractional objectives: a global optimal approach based on the multiparametric disaggregation technique

The majority of data envelopment analysis (DEA) models can be linearized via the classical Charnes–Cooper transformation. Nevertheless, this transformation does not apply to sum-of-fractional DEA efficiencies models, such as the secondary goal I (SG-I) cross efficiency model and the arithmetic mean two-stage network DEA model. To solve a sum-of-fractional DEA efficiencies model, we convert it into bilinear programming. Then, the obtained bilinear programming is relaxed to mixed-integer linear programming (MILP) by using a multiparametric disaggregation technique. We reveal the hidden mathematical structures of sum-of-fractional DEA efficiencies models, and propose corresponding discretization strategies to make the models more easily to be solved. Discretization of the multipliers of inputs or the DEA efficiencies in the objective function depends on the number of multipliers and decision-making units. The obtained MILP provides an upper bound for the solution and can be tightened as desired by adding binary variables. Finally, an algorithm based on MILP is developed to search for the global optimal solution. The effectiveness of the proposed method is verified by using it to solve the SG-I cross efficiency model and the arithmetic mean two-stage network DEA model. Results of the numerical applications show that the proposed approach can solve the SG-I cross efficiency model with 100 decision-making units, 3 inputs, and 3 outputs in 329.6 s. Moreover, the proposed approach obtains more accurate solutions in less time than the heuristic search procedure when solving the arithmetic mean two-stage network DEA model.

Comparison of mixed-integer relaxations with linear and logarithmic partitioning schemes for quadratically constrained problems

Piecewise relaxations are powerful techniques to compute strong dual bounds for (mixed-integer) quadratically constrained problems and provide starting points that can be used by local nonlinear solvers to generate high-quality primal solutions. They work by first partitioning the domain of one of the variables in every quadratic or bilinear term and then selecting the optimal interval in the partition through binary variables. A variety of formulations have been proposed that can be distinguished based on how the number of binary variables scales with the number of intervals. In this paper, we compare linear and logarithmic partitioning schemes that can be derived from the piecewise McCormick relaxation (PCM) or the multiparametric disaggregation technique (MDT). Specifically, we propose MDT for a generic numeric representation system, showing that it becomes a linear partitioning scheme when considering a single digit and varying the basis, and a logarithmic partitioning scheme when fixing the basis and varying the number of digits. Through the solution of 25 benchmark instances for the pooling problem, considering relaxations for the p-, q- and tp-formulations, we show that it is better to rely on the linear scheme from PCM for a small number of intervals and on the logarithmic scheme from base-2 MDT when the number is large. The results also show that significantly stronger dual bounds can be obtained compared to commercial global optimization solvers BARON and GloMIQO.

Estimation of Hourly Flood Hydrograph from Daily Flows Using Artificial Neural Network and Flow Disaggregation Technique

Flood data on a high temporal scale are required for the design of hydraulic structures, flood risk assessment, flood protection, and reservoir operations. Such flood data are typically generated using rainfall-runoff models through an accurate calibration process. The data also can be estimated using a simple relationship between the daily and the sub-daily flow records as an alternative to rainfall–runoff modelling. In this study, we propose an approach combining an artificial neural network (ANN) model for peak flow estimation and the steepness index unit volume flood hydrograph (SIUVFH) method for sub-daily flow disaggregation to generate hydrographs on an hourly time scale. The SIUVFH method is based on the strong relationship between the flood peak and the steepness index, which is defined as the difference between the daily flood peak and daily flow several days before the peak; it is also used for selecting a reference unit volume flood hydrograph to be scaled to obtain the sub-daily flood hydrograph. In this study, to improve the applicability of the SIUVFH method for locations with a weak relationship between the flood peak and steepness index, the ANN-based flood peak estimation was used as an additional indicator to determine a reference unit volume flood hydrograph. To apply the proposed method, ANN models for estimating the peak flows from the mean daily flows during peak and adjacent days were constructed for the studied dam sites. The optimal ANN structures were determined through Monte Carlo cross-validation. The results showed a good performance with statistical measurements of relative root mean square errors of 0.155–0.224, 0.208–0.301, and 0.244–0.382 for the training, validation, and testing datasets, respectively. An application of the combined use of the ANN-based peak estimation and the SIUVFH-based flow disaggregation revealed that the disaggregated hourly flows satisfactorily matched the observed flood hydrograph.

Refinery-wide planning operations under uncertainty via robust optimization approach coupled with global optimization

The paper focuses on the refinery-wide planning operations under uncertainties in product demand and price via robust optimization framework coupled with global optimization. Industrial/simulation data sets are used to build explicit nonlinear surrogate models for correlating the yield/property of products and the operational conditions of secondary processing units. A large-scale nonconvex mixed integer nonlinear programming (MINLP) model is formulated. The product demand and sale price uncertainties are incorporated into the proposed model via the “interval” and “interval + ellipsoidal” uncertainty set, respectively, to obtain the robust counterpart optimization model. A global optimization framework which combines the enhanced normalized multiparametric disaggregation technique (ENMDT) and the optimality-based bound tightening is developed. The proposed model and the global optimization algorithm are applied to an industrial refinery site with three illustrative cases. The results show the value of formulating nonlinear models for secondary processing units and the benefits from applying ENMDT-based algorithm.

A Nonparametric Stochastic Approach for Disaggregation of Daily to Hourly Rainfall Using 3-Day Rainfall Patterns

As infrastructure and populations are highly condensed in megacities, urban flood management has become a significant issue because of the potentially severe loss of lives and properties. In the megacities, rainfall from the catchment must be discharged throughout the stormwater pipe networks of which the travel time is less than one hour because of the high impervious rate. For a more accurate calculation of runoff from the urban catchment, hourly or even sub-hourly (minute) rainfall data must be applied. However, the available data often fail to meet the hydrologic system requirements. Many studies have been conducted to disaggregate time-series data while preserving distributional statistics from observed data. The K-nearest neighbor resampling (KNNR) method is a useful application of the nonparametric disaggregation technique. However, it is not easy to apply in the disaggregation of daily rainfall data into hourly while preserving statistical properties and boundary continuity. Therefore, in this study, three-day rainfall patterns were proposed to improve reproducible ability of statistics. Disaggregated rainfall was resampled only from a group having the same three-day rainfall patterns. To show the applicability of the proposed disaggregation method, probability distribution and L-moment statistics were compared. The proposed KNNR method with three-day rainfall patterns reproduced better the characteristics of rainfall event such as event duration, inter-event time, and toral amount of rainfall event. To calculate runoff from urban catchment, rainfall event is more important than hourly rainfall depth itself. Therefore, the proposed stochastic disaggregation method is useful to hydrologic analysis, particularly in rainfall disaggregation.

Dimension Reduction Based Non-Parametric Disaggregation for Dependence Modeling in Composite System Reliability Evaluation

The correlated variation of bus loads and wind powers has posed a challenge to power system security. How to construct an accurate dependence model becomes vital for power system reliability evaluation. Most currently used methods in estimating the multivariate probability density function (PDF) not only ignore the aggregation constraint but also encounter the curse of dimensionality. In this paper, a new non-parametric disaggregation (ND) technique is presented to disaggregate an aggregate variable (e.g. system load) into non-aggregate variables (e.g. bus loads) while meeting the aggregation constraint. An innovative hierarchical dimension reduction technique is proposed and integrated into the ND model to transform a high-dimensional conditional PDF estimation into some low-dimensional estimations, which effectively resolves the difficulty associated with high dimensionality. The above two techniques are incorporated into an effective three-stage conditional sampling method to form the proposed dimension reduction based ND model (DRND) for high-dimensional dependence modeling in composite system reliability evaluation. The validity of the proposed DRND is verified by three test systems, i.e. MRTS79, MRTS96, and the latest RTS-GMLC.

Trainingless multi-objective evolutionary computing-based nonintrusive load monitoring: Part of smart-home energy management for demand-side management

Electricity is the most widely used form of energy in modern society. One method of satisfying the continuously increasing industrial, commercial, and residential electrical-energy demands of consumers in smart grids is to use an Internet-of-things (IoT) service-oriented electrical-energy management system (EMS) to intrusively monitor and manage electrical loads, which can effectively react to demand-response schemes for demand-side management (DSM). Nonintrusive load monitoring (NILM), a viable cost-effective load disaggregation technique, has recently gained considerable attention as a nonintrusive alternative to EMS in the research field of smart grids. This paper presents a smart IoT-oriented home EMS founded on trainingless multi-objective evolutionary computing-based NILM for DSM in a smart grid. Evolutionary computing-based NILM is considered and addressed as a multi-objective combinatorial optimization problem. The proposed NILM technique can determine the electrical appliances based on their individual electrical characteristics extracted from composite electrical-load consumption with no intrusive deployment of smart plugs or power meters. A fully nonintrusive NILM alternative is considered and proposed. In addition, this alternative is different from conventional NILM because conventional NILM considers artificial intelligence including artificial neural networks (NNs) and deep NN as load classifiers of NILM where training and retraining stages and a hyperparameter tuning procedure are required. The proposed smart IoT-oriented home EMS was experimentally investigated with the trainingless multi-objective evolutionary computing-based NILM in a real house environment. The experimental results confirm that the proposed methodology is feasible.

Evaluation of Candidate Sites in a Proposal for Sustainable Development: “the Gold Route”, Nariño, Colombia

The candidate sites to be included in a sustainable development proposal, called “The Gold Route” to be developed in the municipality of El Tambo, Narino Colombia, were evaluated. This route combines the geological heritage and mining heritage along with the richness in biodiversity, history, and existing culture in the territory, with the use of the technique of participatory action research (PAR), participation planning, and the component disaggregation technique. The studied area is part of what IS known as “Pastos knots” (Nudo de los Pastos), considered along side the Colombian Massif, as two of the most relevant formations of the Andes Mountain Range and a hot spot for biodiversity; furthermore, it is accompanied by important deposits of gold mining that have been exploited since the pre-Columbian era with different methods of extraction and mineral benefits. Of the natural and anthropic processes that have taken place in this territory, patrimonial interest sites have been generated to be incorporated into a proposal for sustainable development: for its identification and prioritization, four main evaluations were developed: (1) assessment by the local community, (2) assessment of attributes and characteristics, (3) assessment of the visual quality of the landscape, and (4) assessment of the preferences of potential visitors. This is the evaluation outcome: (a) 15 sites evaluated, prioritized, and grouped into three phases for implementation, (b) a methodology that managed to integrate different forms of data collection and information analysis in a single document, and (c) a process and its results owned by the targeted community and the local municipal administration.

ITER: A Quarterly Indicator of Regional Economic Activity in Italy

This work documents the construction of the new quarterly indicator of regional economic activity (Indicatore Trimestrale dell’Economia Regionale—ITER), which uses a parsimonious set of regional variables and combines them by means of temporal disaggregation techniques to obtain a quarterly index that is consistent with the official data on national and regional GDP and marked by a small lag compared with the reference period. The methodology was implemented to produce quarterly indicators for the economies of Italy’s four macro-areas in the period 1995–2017. With a view to assessing the performance of the quarterly indicator, a forecasting exercise was conducted regarding annual GDP growth in the four macro-areas for the period 2014–17. The forecasting performance of ITER is in line with that of the indicators developed by other national research institutions.

Tight convex relaxation for TEP problem: a multiparametric disaggregation approach

In recent years, there has been an increasing interest in using AC power flow equations for the transmission expansion planning (TEP) studies. The AC power flow equations are quadratic and hence the TEP problem can be formulated as a mixed-integer quadratically constrained programme. Therefore, the complexity of the TEP problem lies in the non-convexity of AC power flow equations in which the global optimal solution is not guaranteed to be found. This study aims at proposing a tight convex relaxation for the TEP problem. In this context, first, the TEP problem is formulated as a mixed-integer bilinear problem by representing the complex bus voltage in its rectangular coordinates. Second, the multiparametric disaggregation technique (MDT) and piecewise McCormick relaxation are employed to generate a mixed-integer linear relaxation. MDT is based on the discretisation of the domain of one of the variables in every bilinear term. The method presented is much more precise compared with the DC or other linearisation approaches, while the optimal solution is of high quality. The results of the case studies show the tractability and exactness of the proposed model as well as its superiority over the state-of-the-art schemes.

Optimal design and operation of refinery hydrogen systems under multi-scale uncertainties

Robust design of hydrogen systems for large-scale refineries under multi-scale uncertainties has great potential for energy saving and cost reduction. The operation of hydrogen network is greatly affected by the rapidly changing market, deteriorating crude feedstock and varying operating conditions. In this paper, we propose a two-stage stochastic programming with flexibility constraints modeling framework that accounts for multi-scale uncertainties in an integrated strategy, aiming to achieve a good trade-off between the economic performance and the robustness of operation. A tailored-made global optimization algorithm based on multiparametric disaggregation technique is developed to tackle the computational challenge of the resulting MINLP problem with numerous bilinear terms. To illustrate the applicability of the proposed modeling framework and solution algorithm, studies on the optimal design of an industrial hydrogen system are presented. Multi-scenario operation strategy increases the operational flexibility and greatly reduces the total annualized cost in terms of the value of stochastic solution.

Computing Team-Maxmin Equilibria in Zero-Sum Multiplayer Extensive-Form Games

The study of finding the equilibrium for multiplayer games is challenging. This paper focuses on computing Team-Maxmin Equilibria (TMEs) in zero-sum multiplayer Extensive-Form Games (EFGs), which describes the optimal strategies for a team of players who share the same goal but they take actions independently against an adversary. TMEs can capture many realistic scenarios, including: 1) a team of players play against a target player in poker games; and 2) defense resources schedule and patrol independently in security games. However, the study of efficiently finding TMEs within any given accuracy in EFGs is almost completely unexplored. To fill this gap, we first study the inefficiency caused by computing the equilibrium where team players correlate their strategies and then transforming it into the mixed strategy profile of the team and show that this inefficiency can be arbitrarily large. Second, to efficiently solve the non-convex program for finding TMEs directly, we develop the Associated Recursive Asynchronous Multiparametric Disaggregation Technique (ARAMDT) to approximate multilinear terms in the program with two novel techniques: 1) an asynchronous precision method to reduce the number of constraints and variables for approximation by using different precision levels to approximate these terms; and 2) an associated constraint method to reduce the feasible solution space of the mixed-integer linear program resulting from ARAMDT by exploiting the relation between these terms. Third, we develop a novel iterative algorithm to efficiently compute TMEs within any given accuracy based on ARAMDT. Our algorithm is orders of magnitude faster than baselines in the experimental evaluation.

Disaggregation and discretization methods for formation damage estimation in oil and gas fields: an overview

Formation damage could potentially impede production and injection operations. Hence, characterization and discretization processes of formation damage should be connected to quantification and disaggregation techniques, relying on characterization fundamentals that consider chemical and physical changes in the fluid and rock system through the field productive life. This document presents a review of different disaggregation, quantification and discretization methods for the formation damage estimation in oil and gas fields. This review is mainly divided into three main sections, namely: i) Formation damage diagnosis, ii) Formation damage quantification, and iii) Formationdamage disaggregation. This document will aid in the alignment of the academic and industrial sectors to incentivize the prevention and inhibition of formation damage, as well as the optimal design of remediation mechanisms.

Comprehensive analysis of alternative downscaled soil moisture products

Recent advances in L-band passive microwave remote sensing provide an unprecedented opportunity to monitor soil moisture at ~40 km spatial resolution around the globe. Nevertheless, retrieval of the accurate high spatial resolution soil moisture maps that are required to satisfy hydro-meteorological and agricultural applications remains a challenge. Currently, a variety of downscaling, otherwise known as disaggregation techniques have been proposed as the solution to disaggregate the coarse passive microwave soil moisture into high-to-medium resolutions. These techniques take advantage of the strengths of both the passive microwave observations of soil moisture having low spatial resolution and the spatially detailed information on land surface features that either influence or represent soil moisture variability. However, such techniques have typically been developed and tested individually under differing weather and climate conditions, meaning that there is no clear guidance on which technique performs the best. Consequently, this paper presents a quantitative assessment of the existing radar-, optical-, radiometer-, and oversampling-based downscaling techniques using a singular extensive data set collected specifically for that purpose, being the Soil Moisture Active Passive Experiment (SMAPEx)-4 and -5 airborne field campaigns, and the OzNet in situ stations, to determine the relative strengths and weaknesses of their performances. The oversampling-based soil moisture product best captured the temporal and spatial variability of the reference soil moisture overall, though the radar-based products had a better temporal agreement with airborne soil moisture during the short SMAPEx-4 period. Moreover, the difference between temporal analysis of products against in situ and airborne soil moisture reference data sets pointed to the fact that relying on in situ measurements alone is not appropriate for validation of spatially enhanced soil moisture maps.