A Table-Free Approximate Q-Learning-Based Thermal-Aware Adaptive Routing for Optical NoCs

Abstract

Optical networks-on-chips (NoCs) based on silicon photonics have been proposed as an emerging communication architecture for many-core chip multiprocessors. However, the thermal sensitivity of silicon photonics is one of the major challenges. $Q$ -learning-based adaptive routing has been proposed in related work to mitigate the thermal issue. However, table overhead of the traditional table-based $Q$ -routing would scale up quickly with the increase of network size. In this article, we propose a table-free approximate $Q$ -learning-based thermal-aware adaptive routing to find optimal low-loss paths in the presence of on-chip temperature variations. The simulation results show that the proposed table-free approximate $Q$ -learning-based adaptive routing can converge faster and it can achieve similar optimization effect as compared to the best optimization effect of the traditional table-based $Q$ -routing. The performance gap between the proposed approximation method and the traditional table-based $Q$ -routing expands when the network size increases.