Non-technical Losses Research Articles

IRREGULARIDADES NO SISTEMA DE DISTRIBUIÇÃO DE ENERGIA ELÉTRICA NO AMAZONAS

In current times, climate adversities that impact society are observed, such as heat waves and historic droughts, directly influencing electricity consumption and revealing the impacts of irregular practices in energy consumption. In Amazonas, non-technical losses in low voltage reach approximately 42%, gaining national prominence. This study aims to discuss, reflect on the imperative of highlighting the reality of the state of Amazonas in the context of non-technical losses in the electricity sector. It begins with the characterization of irregularities in energy consumption, using ANEEL reporting metrics. It then discusses the adverse impacts on society related to these practices, establishing connections between climatic conditions and non-technical losses. Finally, postures and practices to combat irregularities are outlined, highlighting measures adopted to mitigate and raise awareness among consumers in the Amazon region.

Knowledge distillation-based abnormal power consumption pattern detection for edge environment

Abstract Abnormal power consumption pattern detection can effectively reduce the non-technical loss of electric energy, which is of great significance in maintaining the safe operation of the power grid and protecting the rights and interests of normal users. However, for edge environments with limited communication conditions, uploading electricity consumption data to the cloud for centralized computation is prone to problems such as network congestion and transmission delays. For this reason, this paper proposes a knowledge distillation (KD)-based anomalous power consumption pattern detection model that can be deployed at edge computing terminals. First, the ResNet50 network is selected as the teacher model, and the self-attention mechanism (SAM) is introduced to improve the feature extraction capability of long-term dependency. Then, ResNet18 is used as the student model, and the difference between the output of the student model and the output of the teacher model is minimized by optimizing the distillation loss function. Finally, the effectiveness of the proposed method is verified in practice.

Reducing non-technical losses in electricity distribution networks: Leveraging explainable AI and three lines of defence model to manage operational staff-related factors

This study presents a multidisciplinary approach involving Explainable Artificial Intelligence (ExAI) and operational risk management to reduce Non-Technical Losses (NTL) in electricity distribution. It empirically explores how the activities of employees of utility companies contribute to NTL, a phenomenon often overlooked in existing empirical research. An ensemble classification algorithm is used to analyse utility operations data, and the SHAP explainability technique establishes the predictive significance of staff activities for NTL. Subsequently, these staff activities are mapped into risk cells using the BASEL II and III operational risk definitions, and the Three Lines of Defence (3LoD) model is developed for optimizing electricity distribution. The paper makes three original contributions to the literature: first, it empirically links staff operations to NTL; second, it maps NTL causes to Basel II/III operational risk categories; and finally, to the best of the authors’ knowledge, it is the first study to use the 3LoD model for electricity distribution optimization.

NTL-Unet: A Satellite-Based Approach for Non-Technical Loss Detection in Electricity Distribution Using Sentinel-2 Imagery and Machine Learning.

This study introduces an orbital monitoring system designed to quantify non-technical losses (NTLs) within electricity distribution networks. Leveraging Sentinel-2 satellite imagery alongside advanced techniques in computer vision and machine learning, this system focuses on accurately segmenting urban areas, facilitating the removal of clouds, and utilizing OpenStreetMap masks for pre-annotation. Through testing on two datasets, the method attained a Jaccard index (IoU) of 0.9210 on the training set, derived from the region of France, and 0.88 on the test set, obtained from the region of Brazil, underscoring its efficacy and resilience. The precise segmentation of urban zones enables the identification of areas beyond the electric distribution company's coverage, thereby highlighting potential irregularities with heightened reliability. This approach holds promise for mitigating NTL, particularly through its ability to pinpoint potential irregular areas.

Electricity Theft Detection Using Rule-Based Machine Leaning (rML) Approach

Since electricity theft affects non-technical losses (NTLs) in power distribution systems, power companies are genuinely quite concerned about it. Power companies can use the information gathered by Advanced Metering Infrastructure (AMI) to create data-driven, machine learning-based approaches for Electricity Theft Detection (ETD) in order to solve this problem. The majority of data-driven methods for detecting power theft do take usage trends into account while doing their analyses. Even though consumption-based models have been applied extensively to the detection of power theft, it can be difficult to reliably identify theft instances based only on patterns of usage. In this paper, a novel rule-based combined machine learning (rML) technique is developed for power theft detection to address the drawbacks of systems that rely just on consumption patterns. This approach makes use of the load profiles of energy users to establish rules, identify the rule or rules that apply to certain situations, and classify the cases as either legitimate or fraudulent. The UEDAS smart business power consumption dataset's real-world data is used to assess the performance of the suggested technique. Our technique is an innovation in theft detection that combines years of intensive theft tracking with the use of rule-based systems as feature spaces for traditional machine learning models. With an astounding 93% recall rate for the rule-based feature space combination of the random forest classifier, this novel approach has produced outstanding results. The acquired results show a noteworthy accomplishment in the field of fraud detection, successfully detecting fraudulent consumers 77% of the time during on-site examination.

Fraud Detection of the Electricity Consumption by combining Deep Learning and Statistical Methods

An important issue for the electricity distribution companies is the non-technical loss (NTL), also known as electricity fraud. This issue has a significant impact on the economies of all countries in the world. In this context, we studied the problem of the imbalance between the electrical energy invoiced and the electrical energy supplied within the Algerian economic sector. This article presents an approach to detecting electrical fraud using a combination of Long Short-Term Memory (LSTM) and robust Exponential and Holt-Winters Smoothing (EHWS) methods in order to enhance the accuracy and efficacy of fraud detection mechanisms. The proposed approach investigates the fraudulent behaviour of electricity consumers and unfolds in several key phases. In the first step, monthly consumption forecasts are made and the model with the most accurate results is selected. Then, the phase of detection of anomalies in economic meters and detection of cases of fraud by economic customers begins. This phase relies on the robust exponential and Holt-Winters Smoothing methods for uncovering irregular patterns indicative of potential fraud instances. The proposed model was trained and evaluated, and several experiments were carried out using a large dataset of real users from the economic sector. The dataset comprised approximately 2,000 customers and encompassed 14 years of monthly electricity usage in Bejaia, Algeria. The results of the experiments demonstrate promising performance, underscoring the efficacy of our proposed solution in effectively detecting instances of fraud, leading us to conclude that this proposition is robust and can help improve the accuracy of locating abnormal consumer behaviour and increase the company's profits.

Improving Electricity Theft Detection Using Electricity Information Collection System and Customers’ Consumption Patterns

Electricity theft detection (ETD) techniques employed to identify fraudulent consumers often fail to accurately pinpoint electricity thieves in real time. The patterns associated with electricity use are leveraged to identify anomalies indicative of electricity theft. However, challenges in the benchmark ETD include overfitting and a high incidence of false positives (FPs) resulting from incorrect usage patterns formed by considering only electricity consumption patterns without accounting for external factors that contribute to variations in normal consumption patterns. Further investigation is required to precisely detect instances of electricity theft, thereby minimizing nontechnical losses and forecasting future electricity demand to maintain a stable supply. This study employs a master energy meter located on the transformer side to monitor the amount of energy distributed to the region. Zones with a high likelihood of energy theft are detected by comparing the sum of readings from all the smart meters with the readings from the master energy meter installed on the HV of the substation transformer within the same electric feeder. Ensemble XGBoost machine-learning algorithm and K-Means algorithm are used for the classification of malicious and nonmalicious samples and grouping similar types of consumers together, respectively. This makes the proposed model resistant to false-positive rates caused by changes in usage patterns that aren’t done on purpose. Furthermore, energy thieves are identified by detecting anomalies in consumption behavior while maintaining constant internal and external environmental variables. This novel model proposed here mitigates the FP rate found in the present research of electricity usage data. Our approach outperforms support vector machines, convolution neural network, and logistic regression in simulations. Precision, F1-score, recall, Matthews Correlation Coefficient, Receiver Operating Characteristics (ROC)-Area Under The Curve (AUC), and Precision Recal (PR)-Area Under The Curve (AUC) validate our model. The simulation results show that the proposed K-Means-LSTM-XGBoost model improved the classifier’s F1-score, precision, and recall to 93.75%, 95.16%, and 92.38%, respectively. Our model classifies huge time series data with high precision and can be utilized by the utility for real time electricity theft detection.

An Efficient Method for Detecting Abnormal Electricity Behavior

The non-technical losses caused by abnormal power consumption behavior of power users seriously affect the revenue of power companies and the quality of power supply. To assist electric power companies in improving the efficiency of power consumption audit and regulating the power consumption behavior of users, this paper proposes a power consumption anomaly detection method named High-LowDAAE (Autoencoder model for dual adversarial training of high low-level temporal features). High-LowDAAE adds an extra “discriminator” named AE3 to USAD (UnSupervised Anomaly Detection on Multivariate Time Series), which performs the same function as AE2 in USAD. AE3 performs the same function as AE2 in USAD, i.e., it is trained against AE1 to enhance its ability to reconstruct average data. However, AE3 differs from AE2 because the two “discriminators” correspond to the high-level and low-level time series features output from the shared encoder network. By utilizing different levels of temporal features to reconstruct the data and conducting adversarial training, AE1 can reconstruct the time-series data more efficiently, thus improving the accuracy of detecting abnormal electricity usage. In addition, to enhance the model’s feature extraction ability for time-series data, the self-encoder is constructed with a long short-term memory (LSTM) network, and the fully connected layer in the USAD model is no longer used. This modification improves the extraction of temporal features and provides richer hidden features for the adversarial training of the dual “discriminators”. Finally, the ablation and comparison experiments are conducted using accurate electricity consumption data from users, and the results show that the proposed method has higher accuracy.

Hybrid non-technical-loss detection in fog-enabled smart grids

Electricity theft is one of the major factors contributing to non-technical-losses (NTLs) in power distribution networks. NTL fraud includes frauds in which consumers profit unlawfully by manipulating smart meters (SMs), intruding networks, and so forth. This unlawful act not only undermines people’s efforts to conserve energy but also disrupts the regular billing cycle for power utilities, causing financial losses. In order to help utility companies solve the problems of inefficient electricity inspection and irregular power consumption, two NTL Detection schemes are proposed for NTL fraud prediction. Both schemes employed the auto-regressive integrated moving average (ARIMA) and the machine learning technique to predict the consumer behavior fraud pattern efficiently. Furthermore, extensive simulations are conducted on real-world electricity consumption data sets, which show that the proposed schemes outperformed state-of-the-art solutions and achieved an accuracy of 98%, a precision of 98.6%, a recall of 98.2%, an AUC of 97.9%, and an F1 score of 98.4%.

Definition of Regulatory Targets for Electricity Default Rate in Brazil: Proposition of a Fuzzy Inference-Based Model

The current electricity default rates in continental countries, such as Brazil, pose risks to the economic stability and investment capabilities of distribution utilities. This situation results in higher electricity tariffs for regular customers. From a regulatory perspective, the key issue regarding this challenge is devising incentive mechanisms that reward distribution utilities for their operational and investment choices, aiming to mitigate or decrease electricity non-payment rates and avoid tariff increases for regular customers. Despite adhering to the principles of incentive regulation, the Brazilian Electricity Regulatory Agency (ANEEL) uses a methodological approach to define regulatory targets for electricity defaults tied to econometric models developed to determine targets to combat electricity non-technical losses (NTLs). This methodology has been widely criticized by electricity distribution utilities and academics because it includes many ad hoc steps and fails to consider the components that capture the specificities and heterogeneity of distribution utilities. This study proposes a fuzzy inference-based model for defining regulatory default targets built independently of the current methodological approach adopted by ANEEL and aligned with the principles of incentive regulation. An empirical study focusing on the residential class of electricity consumption demonstrated that it is possible to adopt a specific methodology for determining regulatory default targets and that the fuzzy inference approach can meet the necessary premises to ensure that the principles of incentive regulation and the establishment of regulatory targets are consistent with the reality of each electricity distribution utility.

Detection of Non-Technical Losses on a Smart Distribution Grid Based on Artificial Intelligence Models

Non-technical losses (NTL) have been a growing problem over the years, causing significant financial losses for electric utilities. Among the methods for detecting this type of loss, those based on Artificial Intelligence (AI) have been the most popular. Many works use the electricity consumption profile as an input for AI models, which may not be sufficient to develop a model that achieves a high detection rate for various types of energy fraud that may occur. In this paper, using actual electricity consumption data, additional statistical and temporal features based on these data are used to improve the detection rate of various types of NTL. Furthermore, a model that combines both the electricity consumption data and these features is developed, achieving a better detection rate for all types of fraud considered.

Machine learning-based electricity theft detection using support vector machines

Electricity theft is a serious issue that many nations face, especially in developing areas where non-technical losses can make up a significant percentage of the overall losses sustained by utilities. Electricity theft detection (ETD) is a very challenging task because it frequently introduces irregularities in customer electricity consumption patterns. In recent times, machine learning (ML) techniques have been investigated as a potential solution for ETD. In this research, author propose electricity theft detection based on four kernel functions of support vector machines (SVM). The proposed method analyzes the electricity consumption patterns and then predicts the category of the user. The kernel functions utilized includes polynomial, sigmoid, radial basis function (RBF) and linear kernel function. For experimentation and model training, a dataset of Pakistani utility company is used, which contains the electricity consumption information. The results highlight SVM method works well for accurate ETD. The detection accuracy of the various kernel functions of SVM is 83%, 79%, 80%, and 76% for RBF, polynomial, sigmoid, and linear kernel functions, respectively, demonstrating the effectiveness of the proposed SVM-based method for theft detection. By leveraging these ML-based methods, utility companies can strengthen their ability to detect and prevent electricity theft, leading to improved revenue management and dependability of services.

A novel two‐stage method to detect non‐technical losses in smart grids

AbstractNumerous strategies have been proposed for the detection and prevention of non‐technical electricity losses due to fraudulent activities. Among these, machine learning algorithms and data‐driven techniques have gained prominence over traditional methodologies due to their superior performance, leading to a trend of increasing adoption in recent years. A novel two‐step process is presented for detecting fraudulent Non‐technical losses (NTLs) in smart grids. The first step involves transforming the time‐series data with additional extracted features derived from the publicly available State Grid Corporation of China (SGCC) dataset. The features are extracted after identifying abrupt changes in electricity consumption patterns using the sum of finite differences, the Auto‐Regressive Integrated Moving Average model, and the Holt‐Winters model. Following this, five distinct classification models are used to train and evaluate a fraud detection model using the SGCC dataset. The evaluation results indicate that the most effective model among the five is the Gradient Boosting Machine. This two‐step approach enables the classification models to surpass previously reported high‐performing methods in terms of accuracy, F1‐score, and other relevant metrics for non‐technical loss detection.

Artificial Intelligence for Energy Theft Detection in Distribution Networks

The digitization of distribution power systems has revolutionized the way data are collected and analyzed. In this paper, the critical task of harnessing this information to identify irregularities and anomalies in electricity consumption is tackled. The focus is on detecting non-technical losses (NTLs) and energy theft within distribution networks. A comprehensive overview of the methodologies employed to uncover NTLs and energy theft is presented, leveraging measurements of electricity consumption. The most common scenarios and prevalent cases of anomalies and theft among consumers are identified. Additionally, statistical indicators tailored to specific anomalies are proposed. In this research paper, the practical implementation of numerous artificial intelligence (AI) algorithms, including the artificial neural network (ANN), ANFIS, autoencoder neural network, and K-mean clustering, is highlighted. These algorithms play a central role in our research, and our primary objective is to showcase their effectiveness in identifying NTLs. Real-world data sourced directly from distribution networks are utilized. Additionally, we carefully assess how well statistical methods work and compare them to AI techniques by testing them with real data. The artificial neural network (ANN) accurately identifies various consumer types, exhibiting a frequency error of 7.62%. In contrast, the K-means algorithm shows a slightly higher frequency error of 9.26%, while the adaptive neuro-fuzzy inference system (ANFIS) fails to detect the initial anomaly type, resulting in a frequency error of 11.11%. Our research suggests that AI can make finding irregularities in electricity consumption even more effective. This approach, especially when using data from smart meters, can help us discover problems and safeguard distribution networks.

RNN-BiLSTM-CRF based amalgamated deep learning model for electricity theft detection to secure smart grids.

Electricity theft presents a substantial threat to distributed power networks, leading to non-technical losses (NTLs) that can significantly disrupt grid functionality. As power grids supply centralized electricity to connected consumers, any unauthorized consumption can harm the grids and jeopardize overall power supply quality. Detecting such fraudulent behavior becomes challenging when dealing with extensive data volumes. Smart grids provide a solution by enabling two-way electricity flow, thereby facilitating the detection, analysis, and implementation of new measures to address data flow issues. The key objective is to provide a deep learning-based amalgamated model to detect electricity theft and secure the smart grid. This research introduces an innovative approach to overcome the limitations of current electricity theft detection systems, which predominantly rely on analyzing one-dimensional (1-D) electric data. These approaches often exhibit insufficient accuracy when identifying instances of theft. To address this challenge, the article proposes an ensemble model known as the RNN-BiLSTM-CRF model. This model amalgamates the strengths of recurrent neural network (RNN) and bidirectional long short-term memory (BiLSTM) architectures. Notably, the proposed model harnesses both one-dimensional (1-D) and two-dimensional (2-D) electricity consumption data, thereby enhancing the effectiveness of the theft detection process. The experimental results showcase an impressive accuracy rate of 93.05% in detecting electricity theft, surpassing the performance of existing models in this domain.

Smart Grid Security: An Effective Hybrid CNN-Based Approach for Detecting Energy Theft Using Consumption Patterns.

In Internet of Things-based smart grids, smart meters record and report a massive number of power consumption data at certain intervals to the data center of the utility for load monitoring and energy management. Energy theft is a big problem for smart meters and causes non-technical losses. Energy theft attacks can be launched by malicious consumers by compromising the smart meters to report manipulated consumption data for less billing. It is a global issue causing technical and financial damage to governments and operators. Deep learning-based techniques can effectively identify consumers involved in energy theft through power consumption data. In this study, a hybrid convolutional neural network (CNN)-based energy-theft-detection system is proposed to detect data-tampering cyber-attack vectors. CNN is a commonly employed method that automates the extraction of features and the classification process. We employed CNN for feature extraction and traditional machine learning algorithms for classification. In this work, honest data were obtained from a real dataset. Six attack vectors causing data tampering were utilized. Tampered data were synthetically generated through these attack vectors. Six separate datasets were created for each attack vector to design a specialized detector tailored for that specific attack. Additionally, a dataset containing all attack vectors was also generated for the purpose of designing a general detector. Furthermore, the imbalanced dataset problem was addressed through the application of the generative adversarial network (GAN) method. GAN was chosen due to its ability to generate new data closely resembling real data, and its application in this field has not been extensively explored. The data generated with GAN ensured better training for the hybrid CNN-based detector on honest and malicious consumption patterns. Finally, the results indicate that the proposed general detector could classify both honest and malicious users with satisfactory accuracy.

Nontechnical Loss Detection for Metered Customers in Alexandria Electricity Distribution Company Using Support Vector Machine

Non-technical losses (NTL) during transmission and distribution (T&D) of electrical energy is a major problem faced by utility companies which is very difficult to fight and detect. For that, more of power utilities spend thousands dollar for research centres to find efficient methods for detecting and controlling abnormalities. Electricity theft and billing irregularities forms the main portion of NTL. With the introduction of smart meter, the frequency of reporting energy consumption data to the utility company has been increased. Incoming and outgoing energy could be monitored and analyzed. Electricity theft is a complex problem with many parameters to be evaluated before implementing any measures to detect and control that. These parameters include some issues like social, economic, regional, managerial, infrastructural, and corruption. In recent years, several data mining and research studies on fraud detection and prediction techniques have been carried out in the electricity distribution sector. Support vector machine (SVM) technique has dominated the research for classifying data and detecting fraudulent electricity customers. SVM technique has good ability in data mining and data classification. The paper objective is to analyze the metered energy consumption data recorded and predict the pattern or the form of daily user’s energy consumption, then using SVM to classify the data whether normal or theft. The suggested technique is then tested using real data from Alexandria Electricity Distribution Company (AEDC). The proposed technique was able to distinguish between healthy and theft cases.

Potential benefits of distributed generation in the reduction of non-technical losses

Power generation at or near the consumers can affect citizens positively, especially those who belong to social classes with low earnings. This article proposes an analysis of potential economic and technical benefits that the insertion of Distributed Generation (DG) in low-income communities can create, including the reduction of commercial losses, and alternatives to allow viable financing.

Feature Extraction of Risk Group and Electricity Theft by using Electrical Profiles and Physical Data for Classification in the Power Utilities

Non-technical loss (NTL) is one of the problems that has been a major issue in lost revenue for many years. Electricity distributors have attempted to reduce NTL by detecting electricity theft using various methods. Some events are difficult to detect that conventional meters inspection is inadequate. Moreover, many anomaly patterns found are very complex, confusing in identifying or distinguishing what types of electricity customers are at abnormal risk or energy theft that affects NTL. This paper proposes five key feature extraction methods and six classifying electricity customers using supervised learning. The main problem was studied and collected information, including kilowatt meters, electronic meters, TOU meters, and AMR meters, which cover four customer types that were recorded in the Provincial Electricity Authority (PEA) of Thailand. An electrical profile to be extracted for in-depth analysis of the behavior of each type of electricity customer, combined with the information of physical data to help enhance and increase efficiency. All features examined the relationships in each feature using Pearson correlation and handled unbalanced data using random oversampling (ROS). Then, the extracted data has been trained, validated, and tested to classify three classes: normal, risk, and theft, where we evaluate the results with performance metrics. The results show that random forest (RF) outperforms the rest of the classifiers by achieving a precision-recall area under the curve of 90% and a receiver operating characteristic curve of 78%. Significantly, the results were compared to previous studies and benchmark datasets, which revealed that the proposed method gave better results than other techniques.

Exploiting machine learning to tackle peculiar consumption of electricity in power grids: A step towards building green smart cities

AbstractThe increasing demand for electricity in daily life highlights the need for Smart Cities (SC) to use energy efficiently. Both technical and Non‐Technical Losses (NTL), particularly those resulting from electricity theft, present powerful obstacles; NTL alone can reach billions of dollars. Although Machine Learning (ML) based approaches for NTL detection have been embraced by numerous utilities, there is still a lack of thorough analysis of these methods. Limited research exists on NTL identification evaluation criteria and unbalanced data management in the context of SC. This research compares ML algorithms and data balancing methods to optimize electricity consumption detection. The given research applied the 15 ML techniques of Logistic regression, Bernoulli naive Bayes, Gaussian naive Bayes, K‐Nearest Neighbour, perceptron, passive‐aggressive classifier, quadratic discriminant analysis, SGD classifier, ridge classifier, linear discriminant analysis, decision tree, nearest centroid classifier, multi‐nomial naive Bayes, complement naive Bayes and dummy classifier. While SMOTE, AdaSyn, NRAS, and CCR are considered for data balancing. AUC, F1‐score, and seven relevant performance metrics were used for comparison. We have also implemented SHapely Additive exPlanations (SHAP) for feature importance and model interpretation. Results show varying classifier performance with different balancing methods, emphasizing data preprocessing's role in NTL detection for smart grid security.