Q-Thermal: A Q-Learning-Based Thermal-Aware Routing Algorithm for 3-D Network On-Chips

Abstract

The multilayer structure of 3-D network-on-chips (3-D NoC) leads to the unequal thermal conductance between different layers and, as a result, an unbalanced thermal distribution in the chip. This issue leads to the low reliability and performance degradation of 3-D NoCs. To ensure thermal safety, 3-D NoCs require effective cooling methods. Several thermal-aware routing algorithms have been proposed to route packets in a way which helps mitigation of imbalance temperature distribution. To have a thermal-aware path selection and safety thermal distribution, we propose a new thermal-aware routing method based on the Q-learning algorithm (Q-Thermal). In our method, thermal information propagates over the network using ordinary network packets. Thanks to the Q-learning algorithm that keeps always the thermal information of routers up-to-date that causes routers steer packets toward the cooler paths (laterally or vertically). Based on the experimental results, Q-Thermal reduces the heat and temperature of different layers of 3-D NoCs significantly in comparison with the previous state-of-the-art techniques. Q-Thermal leads to a balanced thermal distribution in the stacked layers of the chip and it improves the network performance considerably.