Energy Trading Scheme Research Articles

Demand Side Management and Peer-to-Peer Energy Trading for Industrial Users Using Two-Level Multi-Agent Reinforcement Learning

Over the past few years, with more concerns on energy equity, energy security and environmental sustainability, peer-to-peer (P2P) energy trading market has increasingly attracted attention to energy users. With the rising electricity price and the growing public doubt to unsustainable development, more efforts are given to the integration between P2P energy trading and industrial practice. To date, the current research work has emphasized electricity cost reduction using demandside management (DSM) and P2P energy trading market to promote energy sharing and create a win–win situation for all industrial market participants. However, less attention has been given to evaluating industrial production. With this attention, this paper formulates a cost minimization problem for multiple discrete manufacturers, where DSM and P2P energy trading market are integrated to optimize manufacturing productivity and energy consumption. Moreover, this paper proposes a two-level reinforcement learning (RL) method to obtain the optimal DSM and energy trading strategies for discrete manufacturers. Unlike the existing RL approaches, the proposed method depicts a twolevel learning approach, where the upper-level program suggests a production plan for the manufacturer to improve the cost return, while the lower-level program optimizes the net benefits. Further, to verify the performance of the proposed method, simulations are conducted based on four different case studies, including different weather and load conditions. Numerical results demonstrate that the proposed algorithm provides a better production plan with enhanced learning speed and economic benefits compared to the existing RL approaches.

Energy Trading Strategy of Distributed Energy Resources Aggregator in Day-Ahead Market Considering Risk Preference Behaviors

Distributed energy resources aggregators (DERAs) are permitted to participate in regional wholesale markets in many counties. At present, new market players such as aggregators participate in China’s power market transactions. However, studies related to market trading strategy have mostly focused on centralized wind power and PV generation units. Few studies have been conducted on the decision-making strategies for DERAs in China’s power market. This paper proposes an auxiliary decision-making model for distributed energy systems to participate in the day-ahead market with more reasonable trading strategies. Firstly, the Gaussian mixture model (GMM) is used to deal with the uncertainties of wind power and photovoltaic (PV) output in the distributed energy system. Secondly, the information gap decision theory (IGDT) is used to deal with the uncertainty of price fluctuations in the spot electricity market. Thirdly, according to the different risk preferences of the DERAs facing market price fluctuation, the robust decision model and opportunity decision-making model in the day-ahead market are constructed, respectively. Finally, to deal with the irrational behavior of the DERAs’ perception of “gain” and “loss” with market risks in China’s two-tier market environment, the prospect theory and the marine predator’s algorithm (MPA) are employed to obtain a day-ahead trading decision scheme for DERA. The analyses show that RDES with robust preference can withstand greater price volatility in the day-ahead market; they will reduce the bidding expectations and increase the system operating cost to improve the achievability of the expected revenue. However, DERAs under the opportunity strategy is more inclined to sell electricity to the market and offset system operating costs with revenue. The proposed model can provide strategic reference for DERAs with different risk preferences to bid in day-ahead market and can improve the level of aggregators’ participation in electricity trading.

A deep reinforcement learning-based bidding strategy for participants in a peer-to-peer energy trading scenario

An efficient energy trading strategy is proven to have a vital role in reducing participants’ payment in the energy trading process of the power grid, which can greatly improve the operation efficiency of the power grid and the willingness of participants to take part in the energy trading. Nevertheless, with the increasing number of participants taking part in the energy trading, the stability and efficiency of the energy trading system are exposed to an extreme challenge. To address this issue, an actor-critic-based bidding strategy for energy trading participants is proposed in this paper. Specifically, we model the bidding strategy with sequential decision-making characteristics as a Markov decision process, which treats three elements, namely, total supply, total demand, and participants’ individual supply or demand, as the state and regards bidding price and volume as the action. In order to address the problem that the existing value-based reinforcement learning bidding strategy cannot be applied to the continuous action space environment, we propose an actor–critic architecture, which endows the actor the ability of learning the action execution and utilizes the critic to evaluate the long-term rewards conditioned by the current state–action pairs. Simulation results in energy trading scenarios with different numbers of participants indicate that the proposed method will obtain a higher cumulative reward than the traditional greedy method.

Deep reinforcement learning for energy trading and load scheduling in residential peer-to-peer energy trading market

The popularization of solar generation enables residential households to supply their loads and trade the surplus energy through residential peer-to-peer (P2P) energy trading market. In a residential community with multi-households, households are able to determine demand-side management (DSM) and energy trading strategies to reduce electricity cost. However, residential users often decide market participation biddings according to their experience and guesses on other households’ biddings, and it is difficult to know how far these biddings are from the best possible market biddings achievable by the P2P market trading mechanism. Targeting the maximum cost savings achievable from this P2P market, this paper focuses on how to obtain such a maximum saving under optimal energy trading and DSM strategies using a multi-agent deep reinforcement learning approach to obtain the optimal solution. In this regard, the energy trading and DSM problem for residential households is formulated as a partially observable Markov decision process (POMDP). Then, a reinforcement learning algorithm with decentralized training and execution approach is adopted in the proposed POMDP problem to fit the policy function and value function, which are trained against each other to learn the optimal strategies by trial-and-error. The simulation result validates that by using the proposed multi-agent reinforcement learning method to optimize the P2P energy trading and DSM strategies, the average reward for households can be significantly improved by 8.52% and 14.03%, respectively, compared with other two state-of-the-art deep reinforcement learning methods.

Blockchain and Secure Element, a Hybrid Approach for Secure Energy Smart Meter Gateways.

This paper presents a new hybrid approach that is suitable for application to energy smart meter gateways, based on combining both blockchain and Secure Element (SE) technologies serving the roles of a secure distributed data storage system and an essential component for building a "root of trust" in IoT platforms simultaneously. Blockchain technology alone may not completely secure a transaction because it only guarantees data immutability, while in most cases, the data has to be also secured at the point of generation. The proposed combinational approach aims to build a robust root of trust by introducing the SE, which will provide IoT devices with trusted computed resources. The feasibility of the proposed method is validated by testing three different implementation scenarios, using different Secure Element systems (SES) combined with blockchain and LPWAN communication technologies to encrypt, transmit, and save data. This hybrid approach aids in overcoming the obstructions of using any one technology alone, and its use is demonstrated with a case study for an Energy Smart Metering gateway that enables the implementation of a local Peer to Peer energy trading scheme that is end-to-end secure and decentralized.

A novel prosumer-centric approach for social welfare maximization considering network voltage constraints in peer-to-peer energy markets

With the increase of prosumers having distributed energy resources in peer-to-peer markets, the network voltages may not remain within safe operating levels. Therefore, any peer-to-peer energy trading scheme must also take technical constraints into consideration in addition to economic benefits. This paper proposes a two-stage social welfare maximization approach which also includes network voltage management. A novel concept of limit trading has been introduced to enhance the value prosumers extract from peer-to-peer energy trading. The simplicity and prosumer-centric nature of this concept makes it quite practical for such trading scenarios. The proposed approach has been tested on IEEE-14 bus network and has been compared to two other approaches in three practical scenarios. The results show that it can improve the total social welfare of the community by up to 10 % without causing voltage issues. Moreover, the proposed approach performs better as compared to other methods in indices like Power Export Index and Jain’s Fairness Index.

A Benders decomposition approach for solving a two-stage local energy market problem under uncertainty

New consumer-centric electricity market schemes have emerged as an alternative to manage the energy balancing locally in distribution networks with high distributed energy resources (DERs) penetration. Peer-to-peer (P2P) energy trading has become an attractive scheme for reducing user and distribution system operators (DSO) costs through the energy surplus exchange between the energy community users, where the uncertainty of DERs and the consumption are challenging issues that require suitable managing for a proper market operation. This article presents a novel two-stage stochastic mixed-integer linear programming model to address the scheduling day-ahead problem of an energy community operating under a P2P energy trading scheme. The formulation proposed minimizes the expected community cost considering the network limitations, ensuring fair trading in the local energy market (LEM), and allowing the prosumers to act as buyers or sellers depending on their load consumption and self-generation. In order to reduce the shared information by consumers/prosumers that trade energy, the proposed model is decomposed into a master problem (MP) and subproblems (SPs) to manage the network constraints and the market requirements, respectively. With this purpose, the Benders decomposition approach is implemented using the recently introduced Strengthened Benders cuts to address the binary variables related to the market and battery operation in the SPs. The model and the algorithm are tested in the 69-bus radial distribution system, considering from 3 to 39 agents trading energy to measure the model scalability. In this analysis, the algorithm convergence shows that the proposed methodology reduces the LEM’s shared information without increasing the energy community cost.

Decentralized peer-to-peer energy trading strategy in energy blockchain environment: A game-theoretic approach

With the proposed concept of energy blockchain, peer-to-peer (P2P) energy trading between prosumers is regarded as a potential way for future power market development. However, existing P2P energy trading often has problems such as low trading efficiency and significant communication losses. This paper presents an energy trading strategy in the blockchain environment to solve the problems of P2P energy trading. First, the type of market is determined based on energy supply and demand. Then, a Stackelberg game is used to establish a two-layer model of the leader (upper) and follower (lower) to determine the price. In addition, we propose an energy trading method based on feasible regions. The trading strategy considers the supply and demand of the market, provides an accurate energy transfer signal, and can help energy blockchain achieve better “self-sufficiency”. A case study proves the effectiveness of the strategy. Compared with the existing methods, the strategy proposed in this paper considers the market type and can promote the coordination and complementation of energy in the microgrid, and improve trading efficiency and reduce communication loss without sacrificing privacy. This study demonstrates that P2P power trading has brought benefits to prosumers, has promoted the development of the energy market, and has shown the massive potential of energy blockchain.

A Hierarchical Deep Reinforcement Learning-Based Community Energy Trading Scheme for a Neighborhood of Smart Households

The uncertainties from the distributed energy resources (DERs) and the prosumer’s heterogeneous characteristics bring great challenges to the operation of the community energy market. In this paper, the real-time energy sharing and management in the community market are decomposed into two-layered sub-problems: the household appliance scheduling problem and the internal energy trading pricing problem. A hierarchical deep reinforcement learning (HDRL) based scheme is proposed for the community energy trading with multiple households, containing a two-stage learning process. In the inner stage, a multi-agent deep reinforcement learning (MADRL) based approach is developed to learn the real-time appliances scheduling policy based on the local observations and given internal electricity price in a decentralized way. In the outer stage, a deep reinforcement learning (DRL) based pricing approach is proposed to determine the real-time internal electricity prices based on the participants’ historical net power and external energy supplier’s electricity prices. The scheduling policy and households’ state are not required in the pricing process. Finally, the simulations with real-world datasets demonstrate that the internal energy trading can significantly reduce the prosumers’ daily cost and the control performance of the proposed scheme is superior to the existing studies.

Peer-to-Peer Local Energy Trading With Voltage Management Under Asynchronous Communication

This paper presents a peer-to-peer (P2P) local energy trading strategy in distribution networks with a novel decentralized voltage management scheme based on voltage sensitivity factors. A social welfare maximizing optimization problem is formulated for P2P energy trading, which is solved by an alternating direction method of multipliers (ADMM) algorithm. The proposed approach is especially designed for asynchronous communication between energy producers and consumers, which may be more suitable for short-term market clearings.

Efficient and privacy-preserving decentralized energy trading scheme in a blockchain environment

Blockchain technology provides the possibility for the realization of decentralized energy trading. However, the blockchain confronts privacy risks and efficiency issues, which hinder its application in energy trading. In this paper, to solve this problem, we propose an efficient and privacy-preserving decentralized energy trading scheme in a blockchain environment. We propose an account mapping algorithm to solve the privacy attack problem. We propose a new transaction processing method, which can protect private data and improve execution efficiency by two-layer transaction processing. Finally, we verify the effectiveness of the proposed scheme for privacy protection and trading efficiency of decentralized energy transactions through experiments.

Energy trading scheme based on consortium blockchain and game theory

With the gradual opening of the electricity sales market, distributed energy trading is becoming an important research topic. However, it is not easy to design practical energy trading schemes in distributed scenario. In particular, known distributed energy trading schemes do not address the security of transaction data and the maximization of benefits among all the participants. In this paper, we propose a distributed energy trading scheme based on consortium blockchain and game theory. In our scheme, a peer-to-peer trading platform is constructed to realize direct transactions among all the participants by the property of decentralization in consortium blockchain. The direct transactions greatly reduce operating costs of energy trading, and at the same time, the security of transaction data can be obtained by the cryptographic techniques such as digital signatures and hash functions associated with the underlying blockchain. Moreover, we design an energy transaction matching mechanism by game theory in our scheme. In the matching mechanism, we construct a game model among all the participants and design an equilibrium solving algorithm, which are the key techniques to realize the maximization of benefits among all the participants in energy trading. The security analysis and experimental results show that our scheme can realize the best transaction price and quantity in the transaction matching and has high security in distributed energy transaction scenarios.

Grid-Oriented Coordination Strategy of Prosumers Using Game-theoretic Peer-to-Peer Trading Framework in Energy Community

In this paper, a grid-oriented peer-to-peer (P2P) energy transaction strategy based on game theory that considers participants with diversity in trading and the impact on the system of trading is studied. For this, the optimization problem of prosumers with ESS is defined according to the lifetime cost of the ESS. The proposed energy trading strategy is carried out based on three game theories. Among them, the evolutionary game between buyers, non-cooperative games between sellers, and Stackelberg games between sellers and buyers are considered. When all games are played sequentially and the convergence point is reached, the negotiation is determined and then ends and a contract is made based on trading price and amount. As a novel contribution in the case of the evolutionary game the community manager indirectly provides information on the system so that when evaluating the seller, the effect on the system is applied as a loss change. In addition, when deriving a utility value by payoff, it is set as an increase/decrease ratio compared to the initial value so that the game proceeds regardless of the size of the participants and the defined problem. To verify the effectiveness of the proposed P2P energy trading strategy, a simulation based on MATLAB is performed. It is confirmed that the effect on the grid is applied to the seller's selling price. When the strategy proposed in the P2P energy trading is applied, the increased purchase and selling amount due to the trading increased by 3.41 % and 4.51 %, respectively, he difference between increased puchase and selling amount decreases by about 36 % from 3.181 kW to 2.03 kW, and the loss due to the change in trading amount also decreased by 1.59 %. This means that the grid impact has been reflected in the P2P energy trading, resulting in efficient operation, and it is the reason for grid operators to accept P2P energy trading. In conclusion, the results of the proposed strategy prove that efficiency in energy use is increased by considering the impacts of the power system in P2P energy trading.

Cloud-Assisted Privacy Protection Energy Trading Based on IBS and Homomorphic Encryption in IIoT

The decentralized and tamper-proof features of blockchain technology can solve the problems of low compatibility, poor flexibility, and single point of failure in the traditional Industrial Internet of Things (IIoT). However, the transparency of the blockchain ledger makes the privacy disclosure of users a huge security risk. Given the privacy leakage problem exposed in the existing energy trading scheme based on the blockchain, this paper creatively proposes a privacy protection scheme for IIoT energy trading based on an identity-based signature (IBS) and homomorphic encryption. On the premise of satisfying the transaction traceability and verifiability, this scheme uses IBS technology to provide an anonymous mechanism for energy trading nodes and utilizes Paillier homomorphic encryption to prevent the disclosure of transaction amounts. To meet the high-concurrency and high-throughput energy trading requirements in IIoT, moreover, the proposed scheme combines the off-chain storage with cloud assistance and the off-chain transaction based on PCN to reduce redundant data written into the blockchain and to improve the concurrent trading efficiency, respectively. The security analysis and performance evaluation results show that the proposed scheme can realize the dual privacy protection of identities and transaction amounts in the trading process at the cost of reasonable calculation.

Blockchain and Double Auction-Based Trustful EVs Energy Trading Scheme for Optimum Pricing

Electric vehicles (EVs) have gained prominence in smart transportation due to their unparalleled benefits of reduced carbon footprints, improved performance, and intelligent energy trading mechanisms. These potential benefits have increased EV adoption at massive scales, but energy management in EVs is a critical study problem. The problem is further intensified due to the scarcity of charging stations (CSs) in near EV proximity. Moreover, as energy transactions occur over open channels, it presents critical security, privacy, and trust issues among decentralized channels. To address the open limitations of trusted energy management and optimize the pricing control among EV entities (i.e., prosumers and consumers), the paper proposes a scheme that integrates blockchain and a truthful double auction strategy for trustful EV trading. To address the transaction scalability, we integrate an Interplanetary File System (IPFS) with a double auction mechanism handled through the Remix Smart Contract environment. The double auction leverages an optimal payoff condition between peer EVs. To address the communication latency, we present the scheme at the backdrop of Fifth Generation (5G) networks that minimizes the optimal payoff response time. The scheme is simulated against parameters such as convergence, profit for consumers, computation time, and blockchain analysis regarding node commit latency, collusion attacks, and EV energy consumption. The results indicate the scheme’s viability against traditional (non-blockchain) approaches with high reliability, scalability, and improved cost-efficiency.

A Peer-2-Peer Management and Secure Policy of the Energy Internet in Smart Microgrids

A peer-to-peer (P2P) energy trading (ET) scheme has been presented in numerous scenarios within the industrial Internet of Things (IIoT), such as vehicle-to-grid network, microgrids, and energy harvesting network. These scenarios present security and privacy issues because of the lack of trust and transparency in the energy markets. In this article, a secure ET system named energy blockchain (EBC) is proposed in order to overcome the security issues. An EBC is suitable for P2P ET situations in which trusted intermediaries are not needed. Similarly, a credit-based payment method to help eliminate ET delays due to transaction confirmation delays on the EBC is proposed. In addition, an optimal pricing method according to the Stackelberg game has been suggested for credit-based loans. According to a real dataset, security analysis and numerical outcomes indicate that the suggested blockchain and payment layout based on credit are both effective and safe in IIoT.

An inter-city energy migration framework for regional energy balance through daily commuting fuel-cell vehicles

Spatiotemporal energy interaction and sharing are promising solutions to penetrate renewable energy, enhance grid power stability, and improve regional energy flexibility. However, the current literature is restrained in a small-scale neighborhood level, without considering inter-city energy migration through spatiotemporal complementarity between renewable-abundant regions (like suburb or countryside areas) and demand-shortage regions (like city centers). In this study, the energy interaction boundary is extended from a neighborhood scale to an inter-city scale, to maximize the renewable energy penetration, demand coverage, and reduce regional energy imbalance. This study firstly proposes a holistic framework on inter-city transportation-based energy migration, consisting of a residential community with rooftop photovoltaic systems and electrical batteries, an office building, hydrogen vehicles (HVs), a hydrogen (H2) station, and local power grids, for the energy transmission between building groups in spatially different regions through the daily commuting of HVs. Optimal grid-regulation strategies are thereafter proposed and adopted to stabilize the grid power and reduce energy costs. Parametric analysis on energy trading strategies and prices has been conducted, to improve the participation motivations of different stakeholders. Results indicate that, compared to the reference case with isolated buildings and vehicles, the transportation-based energy migration framework covers 23.2 % of the office energy demand and elevates the community’s renewable self-use ratio from 72.7 % to 98.6 %. Meanwhile, the maximum grid-export power in the renewable-abundant region (suburb residential community) and the annual grid-import power in the demand-shortage region (city-center office) are reduced by up to 86.9 % (from 155.7 to 20.4 kW) and 29.4 % (from 49.0 to 34.6 kW), respectively. Moreover, even considering the fuel cell degradation cost of HVs, the transportation-based energy migration framework reduces the operating costs of the office building and HVs (the H2 cost and the fuel cell degradation cost) by 16.4 % (from $52791.3 to $44154.7) and 1.7 % (from $27172.5 to $26707.4), respectively. Afterward, compared to the reference case, the peak-shaving and load-shaping grid-regulation strategies can decrease the peak grid-export power of the community by about 71.6 % (from 155.7 to 44.2 kW), and the maximum grid-import power of the office by 23.7 % (from 49.0 to 37.4 kW), respectively. Furthermore, the transportation-based energy migration framework is economically feasible, only when the renewable export price for H2 production is 0.07 $/kWh, the onsite-renewable-generated H2 lower than 6.5 $/kg for the HV owners, and the vehicle-to-building electricity lower than 0.3 $/kWh for the office building. This study provides a novel inter-city energy migration framework with hydrogen networks to enhance district energy sharing, improve regional energy balance and reduce carbon emission, together with frontier guidelines on energy trading prices to promote participation motivations from different stakeholders.

Energy Trading Strategy for Heat and Electricity-Coupled Microgrid Based on Cooperative Game

This paper proposes an energy trading framework for a combined heat and power (CHP) microgrid (MG) integrated with a Photovoltaic (PV) generation system. The parties involved in the CHP-MG energy market are a single MG operator (MGO) and multiple users with electric and thermal demand response (DR). The energy trading process between the MGO and users is formulated as a Nash bargaining game, where the users can bargain with the MGO regarding trading volumes and the prices of electricity and heat. The relationship between the Nash bargaining solution (NBS) and social welfare maximum is studied. A distributed optimization algorithm based on the alternating direction method of multipliers (ADMM) is adopted to achieve NBS. Case studies demonstrate the effectiveness of the proposed trading framework, and show the promotion of economic benefits.

Blockchain-Based Peer-to-Peer Transactive Energy Management Scheme for Smart Grid System.

In Smart Grid (SG), Transactive Energy Management (TEM) is one of the most promising approaches to boost consumer participation in energy generation, energy management, and establishing decentralized energy market models using Peer-to-Peer (P2P). In P2P, a prosumer produces electric energy at their place using Renewable Energy Resources (RES) such as solar energy, wind energy, etc. Then, this generated energy is traded with consumers (who need the energy) in a nearby locality. P2P facilitates energy exchange in localized micro-energy markets of the TEM system. Such decentralized P2P energy management could cater to diverse prosumers and utility business models. However, the existing P2P approaches suffer from several issues such as single-point-of-failure, network bandwidth, scalability, trust, and security issues. To handle the aforementioned issues, this paper proposes a Decentralized and Transparent P2P Energy Trading (DT-P2PET) scheme using blockchain. The proposed DT-P2PET scheme aims to reduce the grid’s energy generation and management burden while also increasing profit for both consumers and prosumers through a dynamic pricing mechanism. The DT-P2PET scheme uses Ethereum-blockchain-based Smart Contracts (SCs) and InterPlanetary File System (IPFS) for the P2P energy trading. Furthermore, a recommender mechanism is also introduced in this study to increase the number of prosumers. The Ethereum SCs are designed and deployed to perform P2P in real time in the proposed DT-P2PET scheme. The DT-P2PET scheme is evaluated based on the various parameters such as profit generation (for prosumer and consumer both), data storage cost, network bandwidth, and data transfer rate in contrast to the existing approaches.

Blockchain based energy trading scheme for vehicle-to-vehicle using decentralized identifiers

A blockchain-based energy trading system is a new paradigm of grid infrastructure, which allows that energy purchaser and seller can efficiently exchange the energy through two-way communication. However, because energy trading services are provided through public networks, these systems are vulnerable to potential security breaches. This paper proposed a privacy-preserving blockchain-based energy trading scheme for vehicle-to-vehicle to resolve the security issues of contemporary systems and provide secure energy trading services. The proposed scheme has high efficiency by applying decentralized identifiers and verifiable credentials technologies because the records of energy trading are not stored on blockchain and blockchain is only utilized in the validation of users. After completing the energy trading, the vehicle issues a verifiable credential to the counterpart to prove the legitimacy of the transaction. We also perform informal and formal security analysis to demonstrate its security and achieve secure mutual authentication, confidentiality, and session key security. Furthermore, we implement AVISPA simulation to show that our scheme is resistant to man-in-the-middle and replay attacks. As a result, the proposed scheme can be used in distributed smart grid environments.