Smart Meter Data Research Articles

Deconstruct: A scalable method of as-built heat power loss coefficient inference for UK dwellings using smart meter data

Dwellings in the UK account for about 25% of global energy demand, of which 60% is space heating making this a key area for efficiency improvement. Dwelling UK Energy Performance Certificates (EPC) are currently based on surveyed data, rather than energy use monitoring. The installation of smart meters provides an opportunity to develop an EPC based on in situ dwelling thermal performance.This paper presents ‘Deconstruct’ – a method of estimating the as-built Heat Power Loss Coefficient (HPLC) of occupied dwellings as a measure of thermal performance, using just smart-meter and meteorological data. Deconstruct is a steady-state grey box building model combined with a data processing pipeline and a model fitting method that limits the effects of confounding factors. Smart meter data from 780 UK dwellings from the UK Energy Demand Research Project (EDRP), was used to calculate a median HPLC of 0.28 kW/°C (±15%). The stability of the estimate across multiple years of data with different weather and energy use was demonstrated. Deconstruct was found to be suitable for large scale inference of dwelling thermal properties using the UK's new smart metering data infrastructure.

Energy disaggregation based on smart metering data via semi-binary nonnegative matrix factorization

The information measured by smart electricity meters in households enables consumers to understand the relationship between their behavior and the power consumption. This paper proposes an energy disaggregation method to estimate the individual-appliance power consumption utilizing the total power consumption data collected by smart meters without requiring additional sensors for supporting sustainable demand-side energy management. In addition, in order to enable appliance labeling and highly accurate estimation without submetered data of the target households, the utilization of auxiliary information such as consumer feedback is suggested. This scheme provides users the opportunity to explicitly point out incorrect current disaggregation results so as to improve forthcoming future decomposition accuracy. The accuracy of the proposed energy disaggregation framework is evaluated using real-world power consumption data sets.

Privacy-Aware Smart Metering: Progress and Challenges

The next-generation energy network, the so-called smart grid (SG), promises a tremendous increase in efficiency, safety and flexibility of managing the electricity grid as compared to the legacy energy network. This is needed today more than ever, as the global energy consumption is growing at an unprecedented rate, and renewable energy sources have to be seamlessly integrated into the grid to assure a sustainable human development. Smart meters (SMs) are among the crucial enablers of the SG concept; they supply accurate high-frequency information about users' household energy consumption to a utility provider, which is essential for time of use pricing, rapid fault detection, energy theft prevention, while also providing consumers with more flexibility and control over their consumption. However, highly accurate and granular SM data also poses a threat to consumer privacy as non-intrusive load monitoring techniques enable a malicious attacker to infer many details of a user's private life. This article focuses on privacy-enhancing energy management techniques that provide accurate energy consumption information to the grid operator, without sacrificing consumer privacy. In particular, we focus on techniques that shape and modify the actual user energy consumption by means of physical resources, such as rechargeable batteries, renewable energy sources or demand shaping. A rigorous mathematical analysis of privacy is presented under various physical constraints on the available physical resources. Finally, open questions and challenges that need to be addressed to pave the way to the effective protection of users' privacy in future SGs are presented.

A novel approach for load profiling in smart power grids using smart meter data

Increasing penetration of distributed energy resources, varying load demands and big data of smart meters require new load models to support power system studies. The big data of smart meters and non-linearities in the load demand require the smart meter data to be represented in an alternative way to use in stochastic simulations to enhance processing. This paper proposes a novel method for stochastic load modelling of smart meter data. The approach turns smart meter data to a manageable level by linearizing energy consumption patterns producing energy classifications. A case study, using real world smart meter data, simulated scenarios to prove the robustness and accuracy of the method. The accuracy of results validates the stability and robustness of the approach and model validation provided substantiation for application in probabilistic studies.

Association rule mining based quantitative analysis approach of household characteristics impacts on residential electricity consumption patterns

The comprehensive understanding of the residential electricity consumption patterns (ECPs) and how they relate to household characteristics can contribute to energy efficiency improvement and electricity consumption reduction in the residential sector. After recognizing the limitations of current studies (i.e. unreasonable typical ECP (TECP) extraction method and the problem of multicollinearity and interpretability for regression and machine learning models), this paper proposes an association rule mining based quantitative analysis approach of household characteristics impact on residential ECPs trying to address them together. First, an adaptive density-based spatial clustering of applications with noise (DBSCAN) algorithm is utilized to create seasonal TECP of each individual customer only for weekdays. K-means is then adopted to group all the TECPs into several clusters. An enhanced Apriori algorithm is proposed to reveal the relationships between TECPs and thirty five factors covering four categories of household characteristics including dwelling characteristics, socio-demographic, appliances and heating and attitudes towards energy. Results of the case study using 3326 records containing smart metering data and survey information in Ireland suggest that socio-demographic and cooking related factors such as employment status, occupants and whether cook by electricity have strong significant associations with TECPs, while attitudes related factors almost have no effect on TECPs. The results also indicate that those households with more than one person are more likely to change ECP across seasons. The proposed approach and the findings of this study can help to support decisions about how to reduce electricity consumption and CO2 emissions.

Phase Identification With Incomplete Data

Phase identification is a process to determine which of the three phases a particular house is connected to. The state-of-the-art identification methods usually exploit smart metering data. However, the data sets are not always available and the major challenge is hence to identify phases with incomplete data set. This paper proposes a novel spectral and saliency analysis identification method to overcome this hurdle. Spectral analysis is first performed to extract the high-frequency features from the incomplete data. Saliency analysis is then adopted to extract salient features from the variations of high-frequency loads in the time domain. Correlation analysis between customer features and the phase features is used to determine customers' phase connectivity. The method is executed iteratively until all customers with smart meters have been allocated to a specific phase or no salient features can be found. It is validated against real data from over 6000 smart meters in Ireland and achieves an accuracy of over 93% with only 10% smart meter penetration ratio in a 100-household network.

Big Data Analytics: Making the Smart Grid Smarter [Guest Editorial

The articles in this special section examine how Big Data analytics is being used in the power industry. Energy and power companies are gradually moving up the analytics ladder, from descriptive analytics, to predictive analytics, and all the way up to prescriptive analytics. Meanwhile, many companies have accumulated large volumes of data from various sources for advanced analytics. Now big data analytics, or applying analytics to big data, has become an emerging field. The topics include data visualization, system operations and planning, energy markets, renewable energy integration, and education. This issue also covers a variety of data sources, such as household- level smart-meter data, synchrophasor data, distribution outage data, and high-resolution weather data. Data visualization is a crucial step in the analytics workflow. Since a picture is worth a thousand words, carefully designed image displays can help analysts identify the proper predictive models, offer comprehensive information to decision makers, and ease communication with stakeholders.

Design of Smart-Meter data acquisition device based on Cloud Platform

In recent years, the government has attached great importance to ‘Four-Meter Unified’ Project. Under the call of national policy, State Grid is participate in building ‘Four-Meter Unified’ Project actively by making use of electricity information acquisition system. In this paper, a new type Smart-Meter data acquisition device based on Cloud Platform is designed according to the newest series of standards Energy Measure and Management System for Electric, Water, Gas and Heat Meter, and this paper introduces the general scheme, main hardware design and main software design for the Smart-Meter data acquisition device.

Visualizing Big Energy Data: Solutions for This Crucial Component of Data Analysis

Visualization is a crucial component of data analysis. It is always a good idea to plot the data before fitting models, making predictions, or drawing conclusions. As sensors of the electric grid are collecting large volumes of data from various sources, power industry professionals are facing the challenge of visualizing such data in a timely fashion. In this article, we demonstrate several data-visualization solutions for big energy data through three case studies involving smart-meter data, phasor measurement unit (PMU) data, and probabilistic forecasts, respectively.

Economic assessment of photovoltaic battery systems based on household load profiles

Technical advances and decreasing costs of photovoltaic (PV) and battery (B) systems are key drivers for the consumer-prosumer transition in many countries. However, the installation of a photovoltaic-battery (PVB) system is not equally profitable for all consumers. This study systematically assesses how heterogeneity in real-world electricity load profiles affects the optimal system configuration and profitability of PVB systems. To that end, we develop a techno-economic simulation model that optimizes the PVB configuration for given electricity load profiles. The analysis uses real-world energy consumption data from 4190 households and is conducted for current electricity rates and weather conditions in Zurich, Switzerland. To account for future price reductions of PV and PVB systems, we conduct a sensitivity analysis that assesses how different cost scenarios influence optimal system configuration and profitability. Finally, we develop and validate a machine learning algorithm that can predict system profitability based only on a limited set of features and on shorter measurement timeframes of smart-meter data. We find that under the current cost scenario (PV: 2000 €/kWp, B: 1000 €/kWh) and without subsidies, about 40% of the analyzed households reach a positive net present value (NPV) for a PV-system, but only for 0.1% of households is the integration of a battery profitable. Under the most optimistic cost scenario for both technologies (PV: 1000 €/kWp, B: 250 €/kWh), 99.9% of the households benefit from the integration of battery storage into their optimal system configuration, with a mean installed PV power of 4.4 kWp and a mean battery size of 9.6 kWh. In all cost scenarios, system profitability varies considerably between households, even for households with comparable total annual demand, primarily due to the heterogeneity in the load profiles. Thus, being able to identify households for whom the installation is profitable is important. The proposed machine learning algorithm predicts optimal configuration, profitability, self-sufficiency, and self-sufficiency ratios with good accuracy, even when only relatively short timeframes of smart-meter data are available. The results of this study are relevant for households making individual investment decisions as well as for utility companies to more effectively identify and approach relevant customers for the installation of PVB systems. Furthermore, the findings enable policymakers to determine the critical levers for increasing private investments into PVB systems in their region and to predict how future developments like component costs will affect the future diffusion of these systems.

Sustainability in university campus: options for achieving nearly zero energy goals

PurposeThe purpose of this study was to design a renovation plan for a university campus building (Department of Electrical and Computer Engineering) with the aim to achieve nearly zero energy performance, ensuring a low specific demand (lower than 44 kWh/m2) and a high level of on-site renewable generation (equivalent to more than 20 per cent of the energy demand).Design/methodology/approachThe baseline demand was characterized based on energy audits, on smart metering data and on the existing building management system data, showing a recent reduction of the electricity demand owing to some implemented measures. The renovation plan was then designed with two main measures, the total replacement of the actual lighting by LEDs and the installation of a photovoltaic system (PV) with 78.8 kWp coupled with an energy storage system with 100 kWh of lithium-ion batteries.FindingsThe designed renovation achieved energy savings of 20 per cent, with 27.5 per cent of the consumed energy supplied by the PV system. This will ensure a reduction of the specific energy of the building to only 30 kWh/m2, with 42.4 per cent savings on the net-energy demand.Practical implicationsThe designed renovation proves that it is possible to achieve nearly zero energy goals with cost-effective solutions, presenting the lighting renovation and the solar PV generation system a payback of 2.3 and 6.9 years, respectively.Originality/valueThis study innovated by defining ambitious goals to achieve nearly zero energy levels and presenting a design based on a comprehensive lighting retrofit and PV generation, whereas other studies are mostly based on envelope refurbishment and behaviour changes.

A Methodology for Optimal Distributed Storage Planning in Smart Distribution Grids

This paper proposes a methodology for optimal planning of distributed storage systems (DSS) in smart grids with high penetration of renewable sources. The DSS can provide distribution systems with various benefits, including arbitrage gain, reduction in the system losses, system resilience enhancement, reduction in nondispatchable energy curtailment, and peak load shaving. In particular, by alleviating the peak load, the system upgrade time can be deferred to future years, resulting in noticeable financial gains for the system operator. In this paper, the problem of DSS planning for optimizing the discounted economic gain of the system operator is formulated and solved as a mixed-integer convex program. Various economic gains are taken into account and the stochasticity of the loads and renewable generations is accounted for by evaluating the total expected gain. Numerical results for DSS planning on a 69-node, 11-kV smart grid and using real data of smart meters and renewable energy sources are presented and discussed.

De-identification of metering data for smart grid personal security in intelligent CCTV-based P2P cloud computing environment

Various security threats exist in the smart grid environment due to the fact that information and communication technology are grafted onto an existing power grid. In particular, smart metering data exposes a variety of information such as users’ life patterns and devices in use, and thereby serious infringement on personal information may occur. Therefore, we are in a situation where a de-identification algorithm suitable for metering data is required. Hence, this paper proposes a new de-identification method for metering data. The proposed method processes time information and numerical information as de-identification data, respectively, so that pattern information cannot be analyzed by the data. In addition, such a method has an advantage that a query such as a direct range search and aggregation processing in a database can be performed even in a de-identified state for statistical processing and availability.

Exploring the impact of network tariffs on household electricity expenditures using load profiles and socio-economic characteristics

Growing self-generation and storage are expected to cause significant changes in residential electricity utilization patterns. Commonly applied volumetric network tariffs may induce imbalance between different groups of households and their respective contribution to recovering the operating costs of the grid. Understanding consumer behaviour and appliance usage together with socio-economic factors can help regulatory authorities to adapt network tariffs to new circumstances in a fair way. Here, we assess the effects of 11 network tariff scenarios on household budgets using real load profiles from 765 households. Thus we explore the possibly disruptive impact of applying peak-load-based tariffs on the budgets of households when they have been mainly charged for consumed volumes before. Our analysis estimates the change in household network expenditure for different combinations of energy, peak and fixed charges, and can help to design tariffs that recover the costs needed for the sustainable operation of the grid. Home energy generation and storage are expected to alter residential energy usage. Careful tariff design is thus needed to ensure fair distribution of grid operation costs. Using smart-meter data and socio-economic profiles, this study explores the potential impact of different tariffs on household expenditure.

An adaptive framework for real-time data reduction in AMI

In existing Advanced Metering Infrastructure (AMI), data collection intervals for each smart meter (SM) typically vary from 15 to 60 min. If we have 1 million SMs that transmit data every 15 min, these SMs will export 4 million records per hour. This leads to dramatically increasing bandwidth usage, energy consumption, traffic cost and I/O congestion. In this work, we present an adaptive framework for minimizing the amount of data transfer from SMs. The reduction in the framework is forecasting-based; when an SM reading is close to the forecasted value, the SM does not transmit the reading. In order for the framework to be adaptive to the ever-changing pattern of SM data, it is provided with a pool of forecasting methods. A supervised-learning scheme is employed to switch in real-time to the forecasting method most suitable to the current data pattern. The experimental results demonstrate that the proposed framework achieves data reduction rates up to 98% with accuracy 96%, depending on the operational parameters of the framework and consumer behavior (statistical features of SM data).

Do Customers Choose Proper Tariff? Empirical Analysis Based on Polish Data Using Unsupervised Techniques

Individual electricity customers that are connected to low voltage network in Poland are usually assigned to the most common G11 tariff group with flat prices for the whole year, no matter the usage volume. Given the diversity of customers’ behavior inside the same specific group, we aim to propose an approach to assign the customers based on some objective factors rather than subjective fixed assignment. With the smart metering data and statistical methods for clustering we can explore and recommend each customer the most suitable tariff to benefit from lower prices thus generate the savings. Further, the paper applies hierarchical, k-means and Kohonen approaches to assign the customers to the proper tariff, assuming that the customer can gain the biggest expenses reduction from the tariff switch. The analysis was conducted based on the Polish dataset with an hourly energy readings among 197 entities.

Residential photovoltaic self-consumption: Identifying representative household groups based on a cluster analysis of hourly smart-meter data

The on-site generation and direct consumption of electricity, so-called self-consumption, with a combined photovoltaic (PV) and battery storage system is becoming increasingly profitable for private households. The profitability of PV self-consumption system largely depends on the match of PV output and the household’s electricity consumption. In energy system modelling, the household’s consumption behaviour is represented by means of a standard load profile. However, the household sector’s heterogeneity is not reflected in one single profile, and the use of only one load profile results in a misjudgement of the profitability of self-consumption. In this study, we present a set of representative household groups that better represent the heterogeneous residential consumption behaviour. The household groups were compiled through the cluster analysis of smart-meter data based on hourly electricity consumption, using household characteristics as explanatory variables. Between the average load profiles of the groups, significant differences were found. Subsequently to the clustering, self-consumption based on a combined PV and battery system was simulated for each household. We found that the achievable level of self-consumption also differs between the groups, which in turn affect the profitability of the PV and battery systems. A statistical analysis revealed that employment and the presence of children are distinguishing factors for the different types of self-consumers. These results suggest that (i) the residential sector is not well represented by a single standard load profile, particularly so in the context of self-consumption modelling. (ii) Different self-consumer types can be identified through socio-demographic characteristics: We found that unemployed households achieve the highest self-sufficiency rates with an average of 40%, the lowest rates with 30% on average occur within households of educated families. (iii) Although the discrepancies are significant, the effect of these differences on profitability is still limited under the current market conditions.

The Incentive to Overinvest in Energy Efficiency: Evidence from Hourly Smart-Meter Data

Many households pay a marginal price for electricity that exceeds the marginal social cost of supplying that electricity. We show evidence that such pricing schemes can create an incentive to overinvest in energy efficiency. Using hourly smart-meter data for households facing time-invariant increasing block prices, we estimate how air conditioner upgrades affect electricity use. We find that the average participating household reduces consumption by 5%, which provides private savings in the form of lower electricity bills and social cost savings by decreasing generation and pollution costs. The private savings exceed the social savings by an average of 140%, so the average household is faced with an incentive to overinvest in energy efficiency. This incentive to overinvest in energy efficiency would be cut in half if consumers faced any one of three alternative pricing plans with lower marginal price but the same average price.

Water End Use Disaggregation Based on Soft Computing Techniques

Disaggregating residential water end use events through the available commercial tools needs a great investment in time to manually process smart metering data. Therefore, it is extremely difficult to achieve a homogenous and sufficiently large corpus of classified single-use events capable of accurately describe residential water consumption. The main goal of the present paper is to develop an automatic tool that facilitates the disaggregation of the individual water consumptions events from the raw flow trace. The proposed disaggregation methodology is conducted through two actions that are iteratively performed: first, the use of an advanced two-step filter, whose calibration is automatically conducted by the Elitist Non-Dominated Sorting Genetic Algorithm NSGA-II; and second, a cropping algorithm based on the filtered water consumption flow traces. As a secondary goal, yet complementary to the main one, a semiautomatic massive classification process has been developed, so that the resulting single-use events can be easily categorized in the different water end uses in a household. This methodology was tested using water consumption data from two different case studies. The characteristics of the households taken as reference and their occupants were unequivocally dissimilar from each other. In addition, the monitoring equipment used to obtain the consumption flow traces had completely different technical specifications. The results obtained from the processing of the two studies show that the automatic disaggregation is both robust and accurate, and produces significant time saving compared to the standard manual analysis.

Privacy-Utility Tradeoff for Applications Using Energy Disaggregation of Smart-Meter Data

Privacy-Utility Tradeoff for Applications Using Energy Disaggregation of Smart-Meter Data