Intelligent Bidding Research Articles

Intelligent strategic bidding in competitive electricity markets using multi-agent simulation and deep reinforcement learning

Aiming at the lack of comprehension of agents in Multi-Agent Simulation (MAS) based on classic Reinforcement Learning algorithms of competitive electricity markets, an intelligent strategic bidding method using Deep Reinforcement Learning (DRL) and MAS is proposed in this paper, which not only can provide more intelligent strategies for market participants to maximize their profits, but can enhance the performance of simulation models dealing with high-dimensional continuous data in electricity markets. Firstly, a theoretical framework of intelligent strategic bidding in competitive electricity markets based on MAS and DRL is proposed, and the process of intelligent bidding in electricity markets based on MAS and DRL is described. Then, three MAS models of intelligent strategic bidding are built based on three classic DRL algorithms, including Deep Q-Network (DQN), Double Deep Q-Network (DDQN), and Asynchronous n-step Q-learning (Async n-step QL), and three algorithms’ convergence speed, computational efficiency, and response sensitivity are compared and analyzed. Finally, a novel Improved Async n-step QL (IAsync n-step QL) algorithm is proposed, the MAS model based on the IAsync n-step QL algorithm for intelligent strategic bidding is established. Simulation results show that the model using the novel DRL algorithm is more profitable and responsive than the classic DRL algorithms.

Recurrent Neural Network-Based Intelligent Bidding Strategy for Profitability Maximization and Control of Electricity Market

Recurrent Neural Network-Based Intelligent Bidding Strategy for Profitability Maximization and Control of Electricity Market

Strategic bidding in a competitive electricity market: An intelligent method using Multi-Agent Transfer Learning based on reinforcement learning

The electricity market will tend to be diverse and competitive to realize Carbon Neutrality goals under Energy Internet. Moreover, bidding strategies and methods are essential for the stable and benign operation of the electricity market. With the development of artificial intelligence and computer simulation technology, multi-agent simulation has gradually become a significant method for electricity market bidding. Among them, Multi-Agent Reinforcement Learning (MARL) can help agents adapt to changing environments. In contrast, Multi-Agent Transfer Learning (MATL) can help agents learn from not only the target task but also other similar tasks. This paper proposes an intelligent strategic bidding theoretical framework in a competitive electricity market using MATL based on MARL and studies four MATL algorithms, including RNN, LSTM, GRU and BGRU. An intelligent bidding simulation model based on the four MATL algorithms is established, and the performance of the intelligent bidding simulation model in the electricity market using the four MATL algorithms based on the MARL Q-learning algorithm is compared and analyzed from the perspective of accuracy and convergence speed. And based on the multi-agent simulation model, examples of bidding strategies are carried out to verify the rationality and effectiveness of the intelligent bidding method using MATL based on MARL.

Actor-critic reinforcement learning for bidding in bilateral negotiation

Designing an effective and intelligent bidding strategy is one of the most compelling research challenges in automated negotiation, where software agents negotiate with each other to find a mutual agreement when there is a conflict of interests. Instead of designing a hand-crafted decision-making module, this work proposes a novel bidding strategy adopting an actor-critic reinforcement learning approach, which learns what to offer in a bilateral negotiation. An entropy reinforcement learning framework called Soft Actor-Critic (SAC) is applied to the bidding problem, and a self-play approach is employed to train the model. Our model learns to produce the target utility of the coming offer based on previous offer exchanges and remaining time. Furthermore, an imitation learning approach called behavior cloning is adopted to speed up the learning process. Also, a novel reward function is introduced that does take not only the agent?s own utility but also the opponent?s utility at the end of the negotiation. The developed agent is empirically evaluated. Thus, a large number of negotiation sessions are run against a variety of opponents selected in different domains varying in size and opposition. The agent?s performance is compared with its opponents and the performance of the baseline agents negotiating with the same opponents. The empirical results show that our agent successfully negotiates against challenging opponents in different negotiation scenarios without requiring any former information about the opponent or domain in advance. Furthermore, it achieves better results than the baseline agents regarding the received utility at the end of the successful negotiations.

Intelligent Bidding Strategies for Prosumers in Local Energy Markets Based on Reinforcement Learning

Local energy markets (LEMs) are proposed in recent years as a way to enable local prosumers and community to trade their electricity and have control over their electrical related resources by ensuring that electricity is traded closer to where it is produced. However, literature is still scarce with the most optimal and effective trading strategies for LEM design. In this work, we propose two reinforcement learning based intelligent bidding strategies for prosumers and consumers trading within an LEM. Our proposed models were evaluated of their performance by testing them in a German real case scenario. The simulation results show that intelligent bidding strategies create additional self sufficiency and market savings to the local community compared to the baseline strategy where the agents make their trading decision randomly without an intelligent agent. Moreover, modelling the intelligent agents to perform towards a common goal creates more share of individual savings for the prosumers and consumers compared to the classical intelligent bidding strategies employed in this work.

Hybrid architecture for intelligent bidding in hourly-based electricity market

Hybrid architecture for intelligent bidding in hourly-based electricity market

Hybrid architecture for intelligent bidding in hourly-based electricity market

Hybrid architecture for intelligent bidding in hourly-based electricity market

Multi-agent simulation for strategic bidding in electricity markets using reinforcement learning

In this paper, Multi-agent Simulation (MAS) and Reinforcement Learning (RL) are studied, a theoretical framework of MAS is proposed for strategic bidding in electricity markets using reinforcement learning, which consists of two parts: one is a MAS system used to simulate the competitive bidding of the actual electricity market; the other is an adaptive learning strategy bidding system used to provide agents with more intelligent bidding strategies. An Experience-Weighted Attraction (EWA) reinforcement learning algorithm (RLA) is applied to the MAS model and a new MAS method is presented for strategic bidding in electricity markets using a new Improved EWA (IEWA). From both qualitative and quantitative perspectives it is compared with other three MAS methods using the Roth-Erev (RE), Q-learning and EWA. The result shows that the performance of the MAS method using IEWA is proved to be better than the others. The four MAS models using four RLAs are built for strategic bidding in electricity markets. Through running the four MAS models, the rationality and correctness of the four MAS methods are verified for strategic bidding in electricity markets using reinforcement learning.

Marketing analytics in the age of machine learning

Machine learning presents unique challenges and tremendous opportunities for today’s marketer, and while many applications have already become common practice, the future holds exciting use cases, some of which are in development and others yet to be imagined. Leveraging the vast amount of data available in the exhaust stream of digital marketing and advertising, and coupling this with almost limitless data storage and processing capacity, the move from rules-based to intelligent analysis is driving efficiencies across a number of marketing initiatives and capabilities. From intelligent bidding and the serving of advertisements across the most common digital channels to advanced segmentation, audience creation, attribution and more, machine learning has already established itself across a large and complex marketing ecosystem. Recent applications in purchase intent and churn modelling, data-driven retargeting and even data-driven creative are using machine learning to provide competitive advantage now and into the future.

Intelligent Bidding in Smart Electricity Markets

This paper has two aims. Firstly, to briefly present overall objectives and expected outcome of an on-going effort concerning design, implementation and the analysis of next generation energy systems based on anticipatory control and a set of ICT emerging technologies and innovations. Secondly, to describe an early proof-of-concept implementation and the associated experimentation of a simulation platform focused on holistic detailed studies of electric energy markets. The proposed platform allows us to elucidate issues related to the open and smart participation of producers and consumers on large-scale e-markets. Based on an existing simulation system, the authors present the required theoretical studies, the enabling technologies, and the practical tools that contribute to the development of such a platform capable of truly large scale simulations. Elements of game theory are utilized to solve the optimization problem related to the maximization of the social welfare of producers and consumers. Selected simulation results associated with the basic required characteristics are presented.

Research on Rapid Bidding of Forgings

Forging, as a traditional machining, plays an important role in Chinese economics construction and equipment production. It is essential to enhance the level of management for forging enterprises. This study aims to reveal the cost composition of forgings and the law of bidding, and thereby make an integrated and intelligent bidding for forgings with reduction of bidding the difficulty and improvements of accuracy. Activity-based costing bidding for forging is studied in depth after analysis of forgings cost composition and enterprise business process. The research promotes the integration and intelligence of forging bidding as well as the accuracy and reasonableness.

Maximizing bidder surplus in simultaneous online art auctions via dynamic forecasting

Abstract This paper presents a novel intelligent bidding system, called SOABER (Simultaneous Online Auction BiddER), which monitors simultaneous online auctions of high-value fine art items. It supports decision-making by maximizing bidders’ surpluses and their chances of winning an auction. One key element of the system is a dynamic forecasting model , which incorporates information about the speed of an auction’s price movement, as well as the level of competition both within and across auctions. Other elements include a wallet estimator , which gauges the bidders’ willingness to pay, and a bid strategizer , which embeds the forecasting model into a fully automated decision system. We illustrate the performance of our intelligent bidding system on an authentic dataset of online art auctions for Indian contemporary art. We compare our system with several simpler ad-hoc approaches, and find it to be more effective in terms of both the extracted surplus and the resulting winning percentage.

SOABER: An Innovative Approach to Maximize Bidder Surplus in Simultaneous Online Art Auctions

This paper presents a novel intelligent bidding system, called SOABER (Simultaneous Online Auction Bidder), which monitors simultaneous online auctions of high-value fine art items. It supports decision making by maximizing bidders’ surplus and their chances of winning an auction. One key element of the system is a dynamic forecasting model, which incorporates information about the speed of an auction's price as well as about the level of competition within and across other auctions. Other elements are a wallet estimator, which gauges bidders' willingness to pay, and a bid strategizer that embeds the forecasting model into a fully automated decision system. We illustrate the performance of our intelligent bidding system on an authentic dataset of online art auctions for Indian contemporary art. We compare our system with several simpler ad-hoc approaches and find it to be more effective in terms of extracted surplus and the resulting winning percentage.

A new reverse auction agent system for m-commerce using mobile agents

In order to get the goods, a buyer must search for the items through several auction sites in Internet. When the auction starts, the buyer needs to connect to these auction sites frequently so that he can monitor the bid states and re-bid. In this paper, we propose an automated negotiation model between two participants, for m-commerce, using collaborative mobile agents called mobile reverse auction agent system (MoRAAS), which mediates between the buyer and the sellers and executes bidding asynchronously and autonomously. This reduces the network load and offers more intelligent bidding. A new double encryption key chain technique and a new RVAP protocol are proposed to achieve unconditional bid privacy. Every losing bidder can control the privacy of their own bids while no trust is needed. Computational cost of our RVAP protocol is reduced by avoiding the costly verifiable encryption technique.

MoCAAS: auction agent system using a collaborative mobile agent in electronic commerce

Abstract To get the items that a buyer wants in an Internet auction, he must search for the items through several auction sites. When the bidding starts, the buyer needs to connect to these auction sites frequently so that he can monitor the bid states and re-bid. A reserve-price auction reduces the number of connections, but this limits the user's bidding strategy. Another problem is equity between the buyer and the seller. Both the buyer and the seller should profit within proper limits. In this paper, we propose an auction agent system using a collaborative mobile agent and a brokering mechanism called MoCAAS (Mobile collaborative auction agent system), which mediates between the buyer and the seller and executes bidding asynchronously and autonomously. This reduces the network load more than with other auction-agents, offers more intelligent bidding, and increases the clear ratio.