Autonomous Energy Management by Applying Deep Q-Learning to Enhance Sustainability in Smart Tourism Cities

Pannee Suanpang,Pitchaya Jamjuntr,Phuripoj Kaewyong,Kittisak Jermsittiparsert

doi:10.3390/en15051906

Abstract

Autonomous energy management is becoming a significant mechanism for attaining sustainability in energy management. This resulted in this research paper, which aimed to apply deep reinforcement learning algorithms for an autonomous energy management system of a microgrid. This paper proposed a novel microgrid model that consisted of a combustion set of a household load, renewable energy, an energy storage system, and a generator, which were connected to the main grid. The proposed autonomous energy management system was designed to cooperate with the various flexible sources and loads by defining the priority resources, loads, and electricity prices. The system was implemented by using deep reinforcement learning algorithms that worked effectively in order to control the power storage, solar panels, generator, and main grid. The system model could achieve the optimal performance with near-optimal policies. As a result, this method could save 13.19% in the cost compared to conducting manual control of energy management. In this study, there was a focus on applying Q-learning for the microgrid in the tourism industry in remote areas which can produce and store energy. Therefore, we proposed an autonomous energy management system for effective energy management. In future work, the system could be improved by applying deep learning to use energy price data to predict the future energy price, when the system could produce more energy than the demand and store it for selling at the most appropriate price; this would make the autonomous energy management system smarter and provide better benefits for the tourism industry. This proposed autonomous energy management could be applied to other industries, for example businesses or factories which need effective energy management to maintain microgrid stability and also save energy.

Highlights

With the advancement of information technology in the disruption era, which is driving digital disruption, the way tourism businesses operate would be transformed by adopting new technology to help support their business operations and elevate them to sustainable development [1–4]
In the simple one-step deep Q-network (DQN), the learning curves showed a large amount of instability, and the remaining algorithms displayed a positive learning process that resulted in reasonable convergence
The results showed this proposed method could save 13.19% in the cost, compared to conducting manual control for energy management

Summary

Introduction

With the advancement of information technology in the disruption era, which is driving digital disruption, the way tourism businesses operate would be transformed by adopting new technology to help support their business operations and elevate them to sustainable development [1–4]. Its three main goals serve as the cornerstone for the researchers’ efforts: (1) ensure that everyone has access to energy services that are affordable, reliable, and contemporary; (2) significantly enhance the amount of renewable energy in the global energy mix; (3) double the global rate of energy efficiency improvement [5]. In this regard, this would see the emergence of using advanced technology for sustaining and managing the energy to go green and preserve the environment, moving toward sustainability [6]. Microgrids have been installed in rural places, towns, and a variety of industries, including commercial, industrial, and military, based on their goals, load types, and geographical and climatic conditions [7]

Objectives

Methods

Results

Conclusion