Incremental reinforcement learning for multi-objective analog circuit design acceleration

Abstract

The design of high-performance analog circuits is a time-consuming process that requires the involvement of highly qualified designers and frequent tedious interactions with circuit simulators. This sets a need for a method that accelerates the circuit design process and reduces unnecessary designer-simulator interactions. Existing methods are either slow or unable to deal with complex tasks. This paper proposes a simulation-based optimization method based on a deep reinforcement learning (DRL) agent to optimize analog circuits and accelerate the design process. In this method, the DRL agent exploits the soft-actor-critic algorithm based on an improved reward function considering multiple conditions to expedite the parameters optimization step of analog circuits. The designer provides the RL agent with the circuit testbench, design requirements and circuit parameters. The agent then repetitively interacts with the circuit simulator to tune these parameters. In this work, an incremental RL approach is also proposed to deal with complex circuits. Incrementality is found to be increasingly beneficial as the circuit complexity increases (increased number of design parameters and competing objectives). The effectiveness of the proposed method was validated on a complex circuit defined by a design space comprising as many as 36 dimensions (17 design variables and 19 competing objectives). The reported results demonstrate that our method can significantly accelerate the design process and yield high-performance optimized designs. Notably, the required number of simulation steps is on the order of 1e3, which is one to two orders of magnitude more efficient than similar state-of-the-art DRL approaches.