Data-Driven Self-Learning Controller Design Approach for Power-Aware IoT Devices based on Double Q-Learning Strategy

Abstract

Operational cycle control is an attractive field of research which can lead to improvements in the services offered by power-aware monitoring embedded IoT devices. Machine learning (ML) is an infrastructure for operational cycle control and provides many approaches which provide more energy-efficient operation. One subfield of ML is Q-learning (QL), which forms the basis of the data-driven self-learning (DDSL) controller. The DDSL algorithm dynamically sets operational duty cycles according to estimates of future collected data values, leading to effective operation of power-aware systems. However, QL performs very poorly in stochastic environments as a result of overestimation of action values. The double estimator implemented in QL therefore applies Double QL (DQL) and forms the basis for a novel Double DDSL (DDDSL). The results of testing a DDDSL controller on historical data showed 42–50 % greater performance than a controller with a fixed duty-cycle, and 2–12 % more performance than a DDSL controller.