Adaptive packet scheduling in IoT environment based on Q-learning

Abstract

In the internet of things (IoT) environment consisting of various devices the traffic condition dynamically changes. Failure to process them in complying with the QoS requirement can significantly degrade the reliability and quality of the system. Therefore, the gateway collecting the data needs to quickly establish a new scheduling policy according to the changing traffic condition. The traditional packet scheduling schemes are not effective for IoT since the data transmission pattern is not identified in advance. Q-learning is a type of reinforcement learning that can establish a dynamic scheduling policy without any prior knowledge on the network condition. In this paper a novel Q-learning scheme is proposed which updates the Q-table and reward table based on the condition of the queues in the gateway. Computer simulation reveals that the proposed scheme significantly increases the number of packets satisfying the delay requirement while decreasing the processing time compared to the existing scheme based on Q-learning with stochastic learning automaton. And the processing time is also minimized by omitting unnecessary computation steps in selecting the action in the iterative Q-learning operations.