Smart Grid Datasets Research Articles

Secure query processing for smart grid data using searchable symmetric encryption

In the last decade, the smart grid has utilized many modern technologies and applications compared with the conventional grid. Cloud computing offers numerous managed services and storage solutions for smart grid data. To balance security, privacy, efficiency, and utility, more efforts should be made to keep up with the rapid evolution of technology. Searchable encryption techniques are widely considered an intelligent solution to ensure data privacy and security while maintaining the functionality to search over encrypted data. In this paper, we propose a more reliable and efficient searchable symmetric encryption scheme for smart grid data. It is a dynamic keyword searchable scheme that uses a symmetric cipher based on a key hashing algorithm (DKS-SCKHA) to generate a secure index. This scheme eliminates the false-positive issue associated with the use of bloom filters and narrows the scope of the retrieved search results. Additionally, it efficiently supports both partial and exact query processing on the encrypted database. Both theoretical and security analyses demonstrate the efficiency and security of the DKS-SCKHA scheme compared to other previous schemes. Comprehensive experiments on a smart grid dataset showed that the DKS-SCKHA scheme is 35–68% more efficient than the schemes compared in this paper. The DKS-SCKHA scheme supports three keyword search scenarios: single, conjunctive, and disjunctive. Furthermore, the DKS-SCKHA scheme is extended to support dynamic fuzzy keyword search on the encrypted database (DEFKS-SCKHA). We evaluated the security and efficiency of the DEFKS-SCKHA scheme through security analysis and experimental evaluation. Theoretical analysis shows that the proposed scheme is secure against known-ciphertext and known-background attacks.

Anomaly Detection in Smart Grid Application

The smart grid, a modernized electrical grid integrating advanced communication and information technologies, enables bidirectional flow of electricity and information, offering numerous benefits such as enhanced reliability, efficiency, and sustainability. However, the complexity and scale of the smart grid introduce new challenges, including the need for effective anomaly detection to ensure its secure and reliable operation. This research paper presents a comprehensive review of state-of-the-art anomaly detection techniques in smart grid applications. We explore various methodologies, including statistical methods, machine learning algorithms, and hybrid approaches, highlighting their strengths and limitations. Furthermore, we discuss the importance of accurate data preprocessing and feature selection to enhance anomaly detection performance. Through a comparative analysis, we demonstrate the effectiveness of different techniques in detecting anomalies in smart grid datasets. Our findings provide valuable insights for researchers and practitioners in developing robust anomaly detection systems for smart grid applications, ultimately contributing to the advancement of smart grid technology.

Knowledge graph‐based multimodal neural networks for smart‐grid defect detection

AbstractThe application of artificial intelligence in smart grid operations, particularly for defect detection, has seen remarkable progress. However, effectively leveraging multi‐modal data sources to enhance defect detection performance remains challenging. This paper presents a novel Knowledge Graph‐based Multimodal Neural Network (KGMNN) approach for robust and efficient defect detection in smart grids. Our proposed method first leverages multiple heterogeneous data sources, such as structural data, temporal sensor readings, and maintenance records, creating a comprehensive representation of the grid system. The system is then modeled as a knowledge graph, with nodes representing various entities and edges signifying their interconnections and interactions. This paper introduces a novel neural network architecture that effectively leverages the knowledge graph structure. This architecture incorporates multi‐modal fusion techniques, enabling the model to process and integrate diverse data sources. It can recognize intricate patterns indicative of potential defects that are often undetectable in individual data modes. Experimental results on several real‐world smart grid datasets demonstrate that the KGMNN significantly outperforms state‐of‐the‐art techniques in terms of precision, recall, and computational efficiency. Furthermore, the KGMNN's ability to provide explainable predictions, aided by the inherent interpretability of the knowledge graph structure, is of significant practical value in the field. This research paves the way for a new generation of AI‐powered defect detection in smart grids, facilitating enhanced grid reliability and operational efficiency.

An intelligent scheduling control method for smart grid based on deep learning.

Nowadays, data analysis is been the most important means to realize power scheduling in smart grids. However, the sharp increase in business data of grids has posed great challenges for this purpose. To deal with such issue, this paper utilizes deep learning to discover hidden rules from massive large-scale big data and particle swarm optimization (PSO) algorithm for generation of control decision. Therefore, an intelligent scheduling control method for smart grid based on deep learning is proposed in this paper. By modeling the historical data of the power company, the long short-term memory algorithm can effectively extract the effective features and realize the prediction of the coal consumption of the unit under certain conditions. At the same time, a kind of intelligent power scheduling algorithm is designed by using PSO, so as to save energy and reduce emissions as much as possible while fulfilling the real-time power generation task. Experiments on a real-world smart grid dataset show that the proposal can achieve a relatively good performance with respect to intelligent scheduling.

DFTMicroagg: a dual-level anonymization algorithm for smart grid data

The introduction of advanced metering infrastructure (AMI) smart meters has given rise to fine-grained electricity usage data at different levels of time granularity. AMI collects high-frequency daily energy consumption data that enables utility companies and data aggregators to perform a rich set of grid operations such as demand response, grid monitoring, load forecasting and many more. However, the privacy concerns associated with daily energy consumption data has been raised. Existing studies on data anonymization for smart grid data focused on the direct application of perturbation algorithms, such as microaggregation, to protect the privacy of consumers. In this paper, we empirically show that reliance on microaggregation alone is not sufficient to protect smart grid data. Therefore, we propose DFTMicroagg algorithm that provides a dual level of perturbation to improve privacy. The algorithm leverages the benefits of discrete Fourier transform (DFT) and microaggregation to provide additional layer of protection. We evaluated our algorithm on two publicly available smart grid datasets with millions of smart meters readings. Experimental results based on clustering analysis using k-Means, classification via k-nearest neighbor (kNN) algorithm and mean hourly energy consumption forecast using Seasonal Auto-Regressive Integrated Moving Average with eXogenous (SARIMAX) factors model further proved the applicability of the proposed method. Our approach provides utility companies with more flexibility to control the level of protection for their published energy data.

A systematic review of big data in energy analytics using energy computing techniques

AbstractIn big data and machine learning, energy analytics has shown rapid development in the past decade. With this development in energy analytics, the energy data has exponentially increased well in different areas and domains. Energy computing techniques have been used in various domains, particularly in energy data to handle the data analysis process. This paper focuses on explaining the concept and evolution of various research questions formulation, data extraction, quality valuation, search strategy, study selection and reporting the results in energy‐oriented domains to evaluate energy‐computing techniques. The thermal power plant, power system and smart grid datasets were highly used out of plenty of other datasets discussed and the ANN algorithm is most commonly used to classify the dataset. Different sets of algorithms are used for data analysis, classification, converting structured data to unstructured data, regression, visualization, and forecasting. Journal publications from the year 2012 to 2020 have reviewed and adopted a systematic approach to identify the findings of all relevant research methodologies in the defined research domain.

Fuzzy Theory Based Quality Assessment of Multivariate Electrical Measurements of Smart Grids

Electrical measurements of smart grids form the backbone of various data driven applications. Proper analysis of these measurements can result in improved system planning, operation, monitoring and protection. However, the efficacy of smart grid data mining is highly influenced by the quality of the data. Hence, quality assessment of smart grid data is essential prior to the usage of the data in various applications. A fuzzy assessment method is proposed in this paper for assessing quality of multivariate electrical measurements of smart grids. At first, relevant quality dimensions of smart grid data are identified. Then, based on certain desirable characteristics, novel membership functions are proposed for assessing the data quality with respect to each of the considered dimensions. The proposed membership functions are evaluated on the current and real power measurements obtained from the power flow analysis of the IEEE 14-bus system. In addition, the proposed method is also implemented on the voltage, current and the real power measurements obtained from the power flow analysis of an actual 34 node feeder located in Arizona. The impact of measurement noise is also investigated by polluting the original measurements with Gaussian noise. It is found that the quality of the noisy measurements worsens with the increase in variance of the added noise. The proposed method has also been validated on a database containing practical SCADA and PMU measurements of the Southern Regional Grid of India. It is found that the PMU datasets are relatively incomplete compared to the SCADA datasets. In addition, the obtained results indicate that PMU data are suitable for use in more number of applications compared to the SCADA datasets. Unlike the existing methods, the proposed method can be used for quantifying the quality of any smart grid dataset that contains electrical measurements of multiple power system variables including boolean variables such as circuit breaker status. Moreover, unlike the existing methods, the proposed method can measure the consistency among the measurements. In addition, the proposed method is found to be sensitive to the distribution of the bad measurements in a given database.

Big Data-Driven Detection of False Data Injection Attacks in Smart Meters

Today’s energy resources are closer to consumers thanks to sustainable energy and advanced metering infrastructure (AMI), such as smart meters. Smart meters are controlled and manipulated through various interfaces in smart grids, such as cyber, physical and social interfaces. Recently, a large number of non-technical losses (NTLs) have been reported in smart grids worldwide. These are partially caused by false data injections (FDIs). Therefore, ensuring a secure communication medium and protected AMIs is critical to ensuring reliable power supply to consumers. In this paper, we propose a novel Big Data-driven solution that employs machine learning, deep learning and parallel computing techniques. We additionally obtained robust statistical features to detect the FDIs based cyber threats at the distribution level. The performance of the proposed model for NTL detection is investigated using private smart grid datasets in the Turkish distribution network for AMI-level cyber threats, and the results are compared to state-of-the-art machine learning algorithms used for NTL classification problems. Our approach shows promising results, as the accuracy, specificity, and precision metrics of most classifiers are above 90% and false positive rates vary between 0.005 to 0.027.

Pairing: Privately Balancing Multiparty Real-Time Supply and Demand on the Power Grid

Microgrids equipped with renewable energy resources have proven to be critical building blocks on the power grid that can greatly improve the grid performance. A promising application would be enabling microgrids to utilize their local energy for further balancing the regional supply and demand at different times – ensuring better system economics and reliability. However, due to the privacy concerns on continuously revealing each microgrid’s local data for deriving real-time optimal balancing decisions, the application of such promising cooperative technique is still limited. In this paper, we design an efficient cryptographic protocol for privately balancing the regional supply and demand, as well as each microgrid’s local supply and demand in real time. We prove the security of our protocol against both passive and active adversaries. Meanwhile, we implemented a prototype of the Pairing system that integrates cryptographic protocol and the power transmission network. We mount the real smart grid datasets into Pairing in real time for system evaluations. The experimental results demonstrate the practicality of our system by scaling to hundreds of microgrids with high accuracy and efficient system performance.

Detecting Prosumer-Community Groups in Smart Grids From the Multiagent Perspective

One of the greatest advancements of the modern era is the evolution of smart grid (SG), which integrates information communication technologies with advanced power electronic technologies to cope with the global energy shortage. The users in SGs are often called the “prosumers,” who not only consume energy but also generate the energy and share it with the utility grid or with other energy consumers. In order to promote sustainable prosumer management in SGs, one of the feasible strategies is to aggregate the prosumers from different locations, but with similar energy behaviors and cohesive interconnections, such groups of prosumers are also called the prosumer-community groups (PCGs). The contribution of this paper is threefold. First, we provide a generalized definition of individual prosumer's energy density, which can be used to detect the underlying leader prosumers in SGs. Second, we formulate the PCG detection (PCG-D) as a multiobjective optimization problem, and present a novel dynamic game model to find the locally Pareto-optimal PCG structure. Third, we propose a partially visible multiagent system (PVMAS), where the viewing angles of both prosumers and PCGs are mutually restricted. The significance of our PVMAS is that it can nicely lead itself to parallelization for PCG-D, due to the fact that the feature updating of each agent is independent of each other. We conduct a series of comprehensive experiments on the simulated SG datasets to validate the performance of PVMAS through comparing it with existing community detection approaches in the literature.

A Secure High-Order CFS Algorithm on Clouds for Industrial Internet of Things

To uncover latent data structures in big data, a high-order clustering algorithm by fast search and find of density peaks has emerged recently, and will bring great application values in industrial Internet-of-Things data management and analysis. With the popularity of cloud computing, it provides users with the convenience of outsourcing calculations while bringing the risk of privacy disclosure. Aiming at the problem above, from the characteristics of the secure cloud service system, this paper proposes a secure high-order clustering algorithm by fast search and find of density peaks on hybrid cloud. In the proposed scheme, the client first builds the encrypted object tensors with user data using homomorphic encryption, then uploads them to the cloud to completely implement the proposed protocols. In the end, clustering results perturbed with random numbers are returned to client for remove perturbation. The performance of the proposed method is evaluated on a smart grid dataset in terms of clustering accuracy, efficiency, and speedup ratio. Experimental results reveal that the proposed approach can accurately and effectively cluster big data without disclosing user privacy while ensuring that the client is very lightweight. Therefore, the proposed scheme with high security and scalability is suitable for clustering industrial Internet-of-Things big data.

A Query-oriented Adaptive Indexing Technique for Smart Grid Big Data Analytics

IoT (Internet of Things) based Smart Grid (SG) is defined as a power grid integrated with a large network of smart objects portrayed by information and communication technology. The data sources of IoT-based SG, as well as their correlations, are usually perplexing, which necessitate indexing techniques for complex queries over the SG dataset to efficiently exploit the rich connotations of data to enable characteristic analytics and fault prediction. As part of popular big data platform, HBase is replacing classic relational data- bases to host huge heterogeneous data records in the form of key-value storage. However, most existing secondary index schemes on HBase are managed and retrieved by corresponding data columns instead of queries to incur inefficiency in answering a complex data query. In this paper, we propose an adaptive indexing technique to speed up a complex data query on HBase for IoT-based SG big data. Our proposed method is based on the observation that most analyses over big power grid data focus on data subsets related to specific power grid events or monitoring data instead of the whole dataset. Theoretical analysis and experimental test show that the proposed query-oriented secondary indexing scheme is feasible in improving the query performance. For a join operation, when compared with a query scheme without secondary indexing, our proposed indexing scheme outperforms from a minimum 6.54 × speedup to a maximum 860 × speedup; when compared with a classic secondary indexing scheme implemented on HBase, our indexing scheme outperforms from a minimum 1.20 × speedup to a maximum 8.68 × speedup. Our indexing technique would be a useful reference for other industrial big data practices.

Data Compression for Next Generation Phasor Data Concentrators (PDCs) in a Smart Grid

The storage space and cost for Smart Grid datasets has been growing exponentially due to its high data-rate of various sensor readings from Automated Metering Infrastructure (AMI), and Phasor Measurement Units (PMUs). The paper focuses on Phasor Data Concentrators (PDCs) that aggregate data from PMUs. PMUs measure real-time voltage, current and frequency parameters across the electrical grid. A typical PDC can process data from anywhere ten to forty PMUs. The paper exploits the need for appropriate security and data compression challenges simultaneously. As a result, an optimal compression method ER1c is investigated for efficient storage of IREG and C37.118 timestamped PDC data sets. We expect that our approach can greatly reduce the storage cost requirements of commercial available PDCs (SEL 3373, GE Multilin P30) by 80%. For example, 2 years of PDC data storage space can be easily replaced with only 10 days of storage space. In addition, our approach in combination with AES 256 encryption can protect PDC data to larger degree as per National Institute of Standards and Technology (NIST) standards.