Electricity Consumption Dataset Research Articles

Hybrid non-technical-loss detection in fog-enabled smart grids

Electricity theft is one of the major factors contributing to non-technical-losses (NTLs) in power distribution networks. NTL fraud includes frauds in which consumers profit unlawfully by manipulating smart meters (SMs), intruding networks, and so forth. This unlawful act not only undermines people’s efforts to conserve energy but also disrupts the regular billing cycle for power utilities, causing financial losses. In order to help utility companies solve the problems of inefficient electricity inspection and irregular power consumption, two NTL Detection schemes are proposed for NTL fraud prediction. Both schemes employed the auto-regressive integrated moving average (ARIMA) and the machine learning technique to predict the consumer behavior fraud pattern efficiently. Furthermore, extensive simulations are conducted on real-world electricity consumption data sets, which show that the proposed schemes outperformed state-of-the-art solutions and achieved an accuracy of 98%, a precision of 98.6%, a recall of 98.2%, an AUC of 97.9%, and an F1 score of 98.4%.

Enhancing hourly electricity forecasting using fuzzy cognitive maps with sample entropy

Accurate electricity consumption forecasting is the crucial to ensure the stable operation of the power system and optimize the smart grid dispatch. However, existing methods may not effectively capture complex potential features and change patterns, and are susceptible to noise, leading to deficiencies such as suboptimal electricity consumption forecasting accuracy, poor stability, and weak generalization capabilities. To fill this gap, a high order fuzzy cognitive map (FCM) forecasting method based on sample entropy and minimax concave penalty (MCP) regularization is proposed. Firstly, two mode decomposition methods are innovatively fused by utilizing sample entropy to extract the complex features of electricity consumption data, then a high order FCM model with powerful system state prediction capability and interpretable knowledge representation is constructed, and finally a FCM learning method based on MCP regularization is proposed. The forecasting method has the merits of low forecasting error, high stability and strong generalization capabilities. Comprehensive experimental results on multiple electricity consumption datasets show that the proposed method achieves optimal performance compared to 10 state-of-the-art baseline methods, demonstrating effectiveness in the task of electricity consumption forecasting, which contributes to more efficient energy management and cost savings, and provides a reliable tool for smart grid planning and management.

A Novel Deep Learning-Based Data Analysis Model for Solar Photovoltaic Power Generation and Electrical Consumption Forecasting in the Smart Power Grid

With the installation of solar panels around the world and the permanent fluctuation of climatic factors, it is, therefore, important to provide the necessary energy in the electrical network in order to satisfy the electrical demand at all times for smart grid applications. This study first presents a comprehensive and comparative review of existing deep learning methods used for smart grid applications such as solar photovoltaic (PV) generation forecasting and power consumption forecasting. In this work, electrical consumption forecasting is long term and will consider smart meter data and socioeconomic and demographic data. Photovoltaic power generation forecasting is short term by considering climatic data such as solar irradiance, temperature, and humidity. Moreover, we have proposed a novel hybrid deep learning method based on multilayer perceptron (MLP), long short-term memory (LSTM), and genetic algorithm (GA). We then simulated all the deep learning methods on a climate and electricity consumption dataset for the city of Douala. Electrical consumption data are collected from smart meters installed at consumers in Douala. Climate data are collected at the climate management center in the city of Douala. The results obtained show the outperformance of the proposed optimized method based on deep learning in the both electrical consumption and PV power generation forecasting and its superiority compared to basic methods of deep learning such as support vector machine (SVM), MLP, recurrent neural network (RNN), and random forest algorithm (RFA).

A multi-step electricity prediction model for residential buildings based on ensemble Empirical Mode Decomposition technique

Residential electricity demand is increasing rapidly, constituting about a quarter of total energy consumption. Electricity demand prediction is one of the sustainable solutions to improve energy efficiency in real-world scenarios. The non-linear and non-stationary consumption patterns in residential buildings make electricity prediction more challenging. This paper proposes a multi-step prediction approach that first conducts cluster analysis to identify seasonal consumption patterns. Secondly, an improved Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN) method and autoencoder model has been deployed to remove irregular patterns, noise, and redundancy from electricity load time series. Finally, the Long Short-Term Memory (LSTM) model has been trained to predict electricity consumption by considering historical, seasonal, and temporal data dependencies. Further, experimental analysis has been conducted on real-time electricity consumption datasets of residential buildings. The comparative results reveal that the proposed multi-step model outperformed the existing state-of-the-art RF-LSTM-based prediction model and attained higher accuracy.

Enhancing Short-Term Electrical Load Forecasting for Sustainable Energy Management in Low-Carbon Buildings

Accurate short-term forecasting of electrical energy loads is essential for optimizing energy management in low-carbon buildings. This research presents an innovative two-stage model designed to address the unique challenges of Electricity Load Forecasting (ELF). In the first phase, robust data preprocessing techniques are employed to handle issues such as outliers, missing values, and data normalization, which are common in electricity consumption datasets in the context of low-carbon buildings. This data preprocessing enhances data quality and reliability, laying the foundation for accurate modeling. Subsequently, an advanced data-driven modeling approach is introduced. The model combines a novel residual Convolutional Neural Network (CNN) with a layered Echo State Network (ESN) to capture both spatial and temporal dependencies in the data. This innovative modeling approach improves forecasting accuracy and is tailored to the specific complexities of electrical power systems within low-carbon buildings. The model performance is rigorously evaluated using datasets from low-carbon buildings, including the Individual-Household-Electric-Power-Consumption (IHEPC) dataset from residential houses in Sceaux, Paris, and the Pennsylvania–New Jersey–Maryland (PJM) dataset. Beyond traditional benchmarks, our model undergoes comprehensive testing on data originating from ten diverse regions within the PJM dataset. The results demonstrate a significant reduction in forecasting error compared to existing state-of-the-art models. This research’s primary achievement lies in its ability to offer an efficient and adaptable solution tailored to real-world electrical power systems in low-carbon buildings, thus significantly contributing to the broader framework of modeling, simulation, and analysis within the field.

Nonlinear pricing in multidimensional context: An empirical analysis of energy consumption

Modern business practices frequently employ a blend of pricing strategies to segment markets effectively. As a result, consumers may encounter pricing schedules that are nonlinear and multidimensional. This paper presents a structural approach for estimating multidimensional nonlinear pricing models involving multiple decision variables in an energy market. Using a unique, rich panel dataset of Chinese household electricity consumption, we structurally estimate consumer preferences under the influence of an Increasing Block Price (IBP) and a Time-of-Use (ToU) system. Our structural approach allows us to distinguish and evaluate household-level price elasticities of demand, presenting a novel explanation for consumers' feedback on marginal price changes. Through model-based simulations, we demonstrate that a 1% increase in price corresponds to a 0.7% reduction in total electricity demand. However, our analysis indicates that practical opportunities for optimization within multi-dimensional pricing systems are limited. Our findings offer distinct insights into the complex interplay between intricate pricing structures and energy consumption behavior, thereby providing valuable guidance for policymakers and regulators.

Community stochastic domestic electricity forecasting

The domestic sector is a significant energy consumer – accounting for around 40% of global electricity demand – due to household demand diversity and complexity. An accurate and robust estimation of domestic electrical loads, environmental impacts, and energy-efficiency potential is crucial for optimal planning and management of energy systems and applications. However, uncertainties resulting from simplistic socio-technical attributes, microclimatic variations, and oversimplification of the effects of interdependent variables make domestic energy modelling challenging. In this research, a hybrid bottom-up community energy forecasting framework is developed to estimate sub-hourly domestic electricity demand using a combination of statistical and engineering modelling approaches by considering key factors influencing household consumption, including demographic characteristics, occupancy patterns, and the features, ownership, and utilisation patterns of electric appliances. The framework is tested on a community in Wales, UK and validated on an annual, daily, and sub-hourly basis with monitored electricity usage averages derived from the UK Energy Follow-Up Survey and the sub-national electricity consumption datasets. Results closely reflect annual and daily household demand at individual dwellings and aggregated levels, with an estimation accuracy of up to 90%. Moreover, the framework facilitates more reliable sub-hourly demand profiles compared to conventional simulation practices that overestimate daily electricity demand and sub-hourly peaks by up to 15% and 50%, respectively.

FIKElectricity: A Electricity Consumption Dataset from Three Restaurant Kitchens in Portugal

Industrial Kitchens are very energy-intensive businesses, consuming between five and seven times more energy per square meter than other commercial spaces like office spaces and retail stores. Still, very little research has been carried out on improving the energy efficiency of this industry. This paper presents the FIKElectricity dataset, a collection of electricity data from three Portuguese restaurant kitchens during their daily operation. The duration of the datasets spans three to four weeks in each industrial kitchen and comprises aggregated and individual appliance consumption, collected at 1 Hz and frac{1}{5} Hz, respectively. The public release of FIKElectricity is expected to draw more attention from the research community to this overlooked industrial sector. The data collection and post-processing methods are thoroughly described in this paper, as well as the dataset organization. Examples showing the dataset’s quality and instructions for its reuse are also provided.

Clean energy consumption in newly poverty-relieved villages: Limited adaptability to external shocks

Billions of people lack access to inexpensive clean energy and live in energy poverty. However, exogenous shocks like pandemics or extremely high temperatures can exacerbate such challenges, particularly in newly poverty-relieved villages. We demonstrate empirically how external shocks can weaken newly poverty-relieved villages' ability to use clean energy—in this case, residential electricity consumption. A household electricity consumption dataset in over 700 poverty-stricken villages spanning from January 2016 to June 2020 has been compiled. The results show that there is a significant socioeconomic heterogeneity in the capacity of electricity consumption in poverty-relieved villages. Specifically, electricity consumption in newly poverty-relieved villages has an increasing trend, while areas with a poorer peripheral economy or a greater proportion of poor households show a muted increase. What's more, such an increase in newly poverty-relieved villages is lacking in strength due to their limited adaptability to extreme temperatures, no matter high or low temperatures. Additionally, compared to their counterparts with no pandemic, the growth of electricity consumption in newly poverty-relieved villages significantly decreased during the national COVID-19 pandemic. This study provides insights for policymakers to adjust policy designs to address the limited adaptability of this vulnerable group to clean energy consumption. This paper adds to the policy discussion of the increased costs of electricity generation due to low carbon transition in energy systems, and the urgency for policy to address this issue to avoid social injustice.

Using machine learning ensemble method for detection of energy theft in smart meters

AbstractElectricity theft is a primary concern for utility providers, as it leads to substantial financial losses. To address the issue, a novel extreme gradient boosting (XGBoost)‐based model utilizing the consumers’ electricity consumption patterns for analysis is proposed for electricity theft detection (ETD). To remove the imbalance in the real‐world electricity consumption dataset and ensure an even distribution of theft and non‐theft data instances, six different artificially created theft attacks were used. Moreover, the utilization of the XGBoost algorithm for classification, especially to identify malicious instances of electricity theft, yielded commendable accuracy rates and a minimal occurrence of false positives. The proposed model identifies electricity theft specific to the regions, utilizing electricity consumption parameters, and other variables as input features. The authors’ model outperformed existing benchmarks like k‐neural networks, light gradient boost, random forest, support vector machine, decision tree, and AdaBoost. The simulation results using the false attacks for balancing the dataset have improved the proposed model's performance, achieving a precision, recall, and F1‐score of 96%, 95%, and 95%, respectively. The results of the detection rate and the false positive rate (FPR) of the proposed XGBoost‐based detection model have achieved 96% and 3%, respectively.

Deep learning-based meta-learner strategy for electricity theft detection

Electricity theft damages power grid infrastructure and is also responsible for huge revenue losses for electric utilities. Integrating smart meters in traditional power grids enables real-time monitoring and collection of consumers’ electricity consumption (EC) data. Based on the collected data, it is possible to identify the normal and malicious behavior of consumers by analyzing the data using machine learning (ML) and deep learning methods. This paper proposes a deep learning-based meta-learner model to distinguish between normal and malicious patterns in EC data. The proposed model consists of two stages. In Fold-0, the ML classifiers extract diverse knowledge and learns based on EC data. In Fold-1, a multilayer perceptron is used as a meta-learner, which takes the prediction results of Fold-0 classifiers as input, automatically learns non-linear relationships among them, and extracts hidden complicated features to classify normal and malicious behaviors. Therefore, the proposed model controls the overfitting problem and achieves high accuracy. Moreover, extensive experiments are conducted to compare its performance with boosting, bagging, standalone conventional ML classifiers, and baseline models published in top-tier outlets. The proposed model is evaluated using a real EC dataset, which is provided by the Energy Informatics Group in Pakistan. The model achieves 0.910 ROC-AUC and 0.988 PR-AUC values on the test dataset, which are higher than those of the compared models.

Temperature change and daily urban-rural residential electricity consumption in northwestern China: Responsiveness and inequality

Daily temperature plays an important role in the comparison of the electricity consumption of urban and rural residents. Previous studies highlighted the impact of income on the relationship between temperature change and residential electricity consumption; however, urban-rural electricity consumption disparity and temperature-electricity response inequality have been neglected. This paper examines the response of urban and rural residents in a China’s northwestern province to temperature changes in terms of electricity consumption, and uncovers its impact on daily electricity consumption inequality between these two groups by using daily households’ electricity consumption dataset from 2017 to 2019. The study results show: (1) The urban response function exhibits a U-shaped distribution, whereas that of rural residents demonstrates a fluctuating upward trend. (2) When the daily mean temperature falls below 24 °C, the gap between urban and rural electricity consumption increases. (3) During the Spring Festival, the negative effects of extremely low temperatures will be reduced, while at the end of October, they will be increased. These findings can help policymakers reduce the electricity consumption inequality between urban and rural areas in the context of climate change.

A deep learning framework using multi-feature fusion recurrent neural networks for energy consumption forecasting

Accurate energy load forecasting can not only provide favorable conditions for ensuring energy security but also reduce carbon emissions and thereby slow down the process of global warming. With the continuous construction of smart grids, energy consumption information can be easily obtained, which provides a good foundation for forecasting work. In the household energy consumption forecasting field, the current mainstream methods combine signal processing with artificial intelligence methods to improve forecasting accuracy. However, due to differences in residents' daily habits and sensor errors, the energy consumption data of individual households exhibit high volatility and contain noise. Existing methods' accuracy, robustness, generalization ability, and efficiency are challenged when considering multi-time scales and multi-samples. Based on this, this paper proposes a deep learning framework using multi-feature fusion recurrent neural networks for univariate energy consumption data. The proposed framework employs improved Singular Spectrum Analysis (SSA) and multivariate input recurrent neural network (RNN). To verify the effectiveness of the proposed framework, a series of comparative experiments were conducted on the Moroccan buildings' electricity consumption dataset (MORED). Experiments reveal that the proposed framework achieves adaptive decomposition of raw time series data and multi-feature fusion input. Moreover, under the evaluation of multiple metrics, the proposed framework outperforms the state-of-the-art methods in terms of forecasting accuracy, robustness, generalization ability, and efficiency.

Machine Learning and Bagging to Predict Midterm Electricity Consumption in Saudi Arabia

Electricity is widely regarded as the most adaptable form of energy and a major secondary energy source. However, electricity is not economically storable; therefore, the power system requires a continuous balance of electricity production and consumption to be stable. The accurate and reliable assessment of electrical energy consumption enables planning prospective power-producing systems to satisfy the expanding demand for electrical energy. Since Saudi Arabia is one of the top electricity consumers worldwide, this paper proposed an electricity consumption prediction model in Saudia Arabia. In this work, the authors obtained a never-before-seen dataset of Saudi Arabia’s electricity consumption for a span of ten years. The dataset was acquired solely by the authors from the Saudi Electrical Company (SEC), and it has further research potential that far exceeds this work. The research closely examined the performance of ensemble models and the K* model as novel models to predict the monthly electricity consumption for eighteen service offices from the Saudi Electrical Company dataset, providing experiments on a new electricity consumption dataset. The global blend parameters for the K* algorithm were tuned to achieve the best performance for predicting electricity consumption. The K* model achieved a high accuracy, and the results of the correlation coefficient (CC), mean absolute percentage error (MAPE), root mean squared percentage error (RMSPE), mean absolute error (MAE), and root mean squared error (RMSE) were 0.9373, 0.1569, 0.5636, 0.016, and 0.0488, respectively. The obtained results showed that the bagging ensemble model outperformed the standalone K* model. It used the original full dataset with K* as the base classifier, which produced a 0.9383 CC, 0.1511 MAPE, 0.5333 RMSPE, 0.0158 MAE, and 0.0484 RMSE. The outcomes of this work were compared with a previous study on the same dataset using an artificial neural network (ANN), and the comparison showed that the K* model used in this study performed better than the ANN model when compared with the standalone models and the bagging ensemble.

Electricity theft detection based on hybrid random forest and weighted support vector data description

Improving the detection rate of electricity theft users in smart grids is crucial to the safe operation of the power system and the economic efficiency of the grid. The traditional electricity theft detection relies on manual household inspection, which is costly and inefficient. With the application of smart meters, machine learning-based methods are gradually applied to detect electricity theft users. However, the training of the model is limited by the unbalanced electricity consumption dataset, and the performance of the model is limited in the process of processing high-dimensional data. Therefore, this paper adopts ensemble learning and imbalance data classification model to analyze the electricity theft detection problem in complex grid environment, and proposes an electricity theft detection model based on random forest and weighted support vector description. Firstly, the feature selection algorithm for mixed load data with statistical features based on random forest (FS-MLDRF) is used to extract the features that can best represent the customers’ electricity consumption behavior, so as to reduce the complexity of user data. Finally, an electricity theft detection based on hybrid random forest and weighted support vector data description is constructed (HFR-WSVDD), and the SVDD classifier is trained with the features weighted by feature importance to improve the performance of power theft detection in the case of unbalanced data. The simulation experiment results on the real electricity dataset show that compared with existing feature selection algorithms and anomaly detection algorithms, the algorithms proposed in this paper have excellent feature representation capability and better classification performance.

Extremely randomised trees machine learning model for electricity theft detection

Electricity ranks among the world’s most plundered commodities. The fraudulent act of acquiring electrical power without paying for it is termed electricity theft. Electricity theft is captured in power distribution systems as non-technical losses (NTL), representing a major loss in revenue for power utility companies. Electricity theft has far-reaching financial consequences owing to unrealised revenue, and this has a knock-on effect on both developed and developing countries because electricity represents a major part of a country’s GDP and facilitates other industries. AMI-based smart energy meters (SM) gather large amounts of electricity consumption (EC) data that power utilities can utilise to monitor and detect fraudulent customers. This EC data is fed to a machine learning (ML) based electricity theft detection model to learn the behaviour of fraudulent customers. However, existing ML-based electricity theft detection (ETD) models do not produce the best outcomes because of; consecutive missing values in EC datasets, data class imbalance problems, inappropriate hyperparameter tuning of ML models, etc. This research introduces an ETD model using an extremely randomised trees classifier to detect electricity theft in smart grids efficiently. SMOTE Tomek sampling is used to deal with the data class imbalance, and the grid search optimisation technique is employed to optimise the hyperparameters of the proposed model. The proposed model shows its capacity to detect electricity theft by obtaining 98%, 95.06%, 98%, 97%, 98%, and 99.65% accuracy, Matthew’s correlation coefficient, detection rate, Precision, F1-score, and area under the curve receiver operating characteristic, respectively.

CAC-WOA: context aware clustering with whale optimization algorithm for knowledge discovery from multidimensional space in electricity application

Energy consumption forecasting is a hot field of research; despite the number of developed models, projecting electric consumption in residential buildings remains problematic owing to the significant unpredictability of occupant energy use behavior. Discovering the electricity consumption knowledge from the multi-dimensional data streams (MDDS) of electricity logs is a challenging research problem. We propose a novel electricity knowledge discovery model proposed from the MDDS using clustering and machine learning. Context-aware clustering with whale optimization algorithm (CAC-WOA) is proposed to discover the predictive features from the electricity MDDS and perform the predictions using WOA. The CAC-WOA consists of two phases context-aware group formation and a WOA-based machine learning predictive model. In the CAC algorithm, group formation using electricity contextual information to estimate the robust predictive features are proposed. Using such predictive features, the predictive model using the WOA-based artificial neural network (ANN) is built. The modified ANN technique using the WOA algorithm is used to reduce the error rates and improve the prediction accuracy. The experimental outcomes using publicly available electricity consumption datasets prove the efficiency of the CAC-WOA model. Overall prediction accuracy is improved by 3.27% and prediction time is reduced by 11.31% using CAC-WOA compared state-of-the-art solutions.

Detecting Nontechnical Losses in Smart Meters Using a MLP-GRU Deep Model and Augmenting Data via Theft Attacks

The current study uses a data-driven method for Nontechnical Loss (NTL) detection using smart meter data. Data augmentation is performed using six distinct theft attacks on benign users’ samples to balance the data from honest and theft samples. The theft attacks help to generate synthetic patterns that mimic real-world electricity theft patterns. Moreover, we propose a hybrid model including the Multi-Layer Perceptron and Gated Recurrent Unit (MLP-GRU) networks for detecting electricity theft. In the model, the MLP network examines the auxiliary data to analyze nonmalicious factors in daily consumption data, whereas the GRU network uses smart meter data acquired from the Pakistan Residential Electricity Consumption (PRECON) dataset as the input. Additionally, a random search algorithm is used for tuning the hyperparameters of the proposed deep learning model. In the simulations, the proposed model is compared with the MLP-Long Term Short Memory (LSTM) scheme and other traditional schemes. The results show that the proposed model has scores of 0.93 and 0.96 for the area under the precision–recall curve and the area under the receiver operating characteristic curve, respectively. The precision–recall curve and the area under the receiver operating characteristic curve scores for the MLP-LSTM are 0.93 and 0.89, respectively.

Electricity Consumption Prediction in an Electronic System Using Artificial Neural Networks

The tremendous rise of electrical energy demand worldwide has led to many problems related to efficient use of electrical energy, consequently posing difficult challenges to electricity consumers of all levels—from households to large companies’ facilities. Most of these challenges could be overcome by the accurate prediction of electricity demand. Additionally, balance responsibility includes the penalty-based financial mechanism causing extra expense for badly estimated consumption, above the allowed imbalance limits. In this paper, a method for electricity consumption prediction based on artificial neural networks is proposed. The electricity consumption dataset is obtained from a cold storage facility, which generates data in hourly intervals. The data obtained are measured for a period of over 2 years and then separated to four seasons, so different models are developed for each season. Five different network structures (ordinary RNN, LSTM, GRU, bidirectional LSTM, bidirectional GRU) for five different values of horizon, i.e., input data (one day, two days, four days, one week, two weeks) are examined. Performance indices, such as mean absolute percentage error (MAPE), root mean square error (RMSE), mean absolute error (MAE) and mean square error (MSE), are used in order to obtain qualitative and quantitative comparisons among the obtained models. The results show that the modifications of recurrent neural networks perform much better than ordinary recurrent neural networks. GRU and LSTMB structures with horizons of 168h and 336h are found to have the best performances.

A Hybrid Deep Learning-Based Model for Detection of Electricity Losses Using Big Data in Power Systems

Electricity theft harms smart grids and results in huge revenue losses for electric companies. Deep learning (DL), machine learning (ML), and statistical methods have been used in recent research studies to detect anomalies and illegal patterns in electricity consumption (EC) data collected by smart meters. In this paper, we propose a hybrid DL model for detecting theft activity in EC data. The model combines both a gated recurrent unit (GRU) and a convolutional neural network (CNN). The model distinguishes between legitimate and malicious EC patterns. GRU layers are used to extract temporal patterns, while the CNN is used to retrieve optimal abstract or latent patterns from EC data. Moreover, imbalance of data classes negatively affects the consistency of ML and DL. In this paper, an adaptive synthetic (ADASYN) method and TomekLinks are used to deal with the imbalance of data classes. In addition, the performance of the hybrid model is evaluated using a real-time EC dataset from the State Grid Corporation of China (SGCC). The proposed algorithm is computationally expensive, but on the other hand, it provides higher accuracy than the other algorithms used for comparison. With more and more computational resources available nowadays, researchers are focusing on algorithms that provide better efficiency in the face of widespread data. Various performance metrics such as F1-score, precision, recall, accuracy, and false positive rate are used to investigate the effectiveness of the hybrid DL model. The proposed model outperforms its counterparts with 0.985 Precision–Recall Area Under Curve (PR-AUC) and 0.987 Receiver Operating Characteristic Area Under Curve (ROC-AUC) for the data of EC.