Identifying optimal cycles in quantum thermal machines with reinforcement-learning

Paolo A Erdman,Frank Noé

doi:10.1038/s41534-021-00512-0

Abstract

The optimal control of open quantum systems is a challenging task but has a key role in improving existing quantum information processing technologies. We introduce a general framework based on reinforcement learning to discover optimal thermodynamic cycles that maximize the power of out-of-equilibrium quantum heat engines and refrigerators. We apply our method, based on the soft actor-critic algorithm, to three systems: a benchmark two-level system heat engine, where we find the optimal known cycle; an experimentally realistic refrigerator based on a superconducting qubit that generates coherence, where we find a non-intuitive control sequence that outperforms previous cycles proposed in literature; a heat engine based on a quantum harmonic oscillator, where we find a cycle with an elaborate structure that outperforms the optimized Otto cycle. We then evaluate the corresponding efficiency at maximum power.

Highlights

Thermal machines convert between thermal and mechanical energy in a controlled manner
Our Reinforcement Learning (RL) approach discovers a new and non-intuitive cycle that outperforms previous proposals48,54,62. (iii) a heat engine based on a harmonic oscillator[42], where we find a cycle with an elaborate structure that shares qualitative similarities with the Otto cycle, but which performs better thanks to additional features
We can control the evolution of the quantum system (QS) and exchange work with it d(t), we expect the RL agent to automatically discover the optimality of periodically driving the Quantum thermal machines (QTMs) and the corresponding through a set of time-dependent control parameters u(t)

Summary

Introduction

Thermal machines convert between thermal and mechanical energy in a controlled manner. Quantum thermal machines (QTMs) perform thermodynamic cycles via nanoscale systems that can be as small as single particles or two-level quantum systems (qubits). Quantum heat engines and refrigerators could find applications in heat management at the nanoscale[1], or for on-chip active cooling[2,3]. Quantum thermodynamics is a rapidly growing research area that aims at the understanding, design, and optimization of QTMs4. An open fundamental question is whether quantum effects can boost the performance of QTMs2,4,5. On the other hand, understanding how to optimally control the nonequilibrium dynamics of open quantum systems is a complicated task, which can improve existing quantum information processing technologies. The heat flow across these systems has been measured[11,12,13,14], and recent experimental realizations of QTMs have been reported[15,16,17,18,19,20,21,22]

Objectives

Methods

Results

Conclusion