Abstract
Differentiable programming is a fresh programming paradigm which composes parameterized algorithmic components and trains them using automatic differentiation (AD). The concept emerges from deep learning but is not only limited to training neural networks. We present theory and practice of programming tensor network algorithms in a fully differentiable way. By formulating the tensor network algorithm as a computation graph, one can compute higher order derivatives of the program accurately and efficiently using AD. We present essential techniques to differentiate through the tensor networks contractions, including stable AD for tensor decomposition and efficient backpropagation through fixed point iterations. As a demonstration, we compute the specific heat of the Ising model directly by taking the second order derivative of the free energy obtained in the tensor renormalization group calculation. Next, we perform gradient based variational optimization of infinite projected entangled pair states for quantum antiferromagnetic Heisenberg model and obtain start-of-the-art variational energy and magnetization with moderate efforts. Differentiable programming removes laborious human efforts in deriving and implementing analytical gradients for tensor network programs, which opens the door to more innovations in tensor network algorithms and applications.
Highlights
Tensor networks are prominent approaches for studying classical statistical physics and quantum many-body physics problems [1,2,3]
We compute the specific heat of the Ising model directly by taking the second-order derivative of the free energy obtained in the tensor renormalization group calculation
We first review the core idea of automatic differentiation and explain its application to various tensor network contraction algorithms formulated in terms of computation graphs
Summary
Tensor networks are prominent approaches for studying classical statistical physics and quantum many-body physics problems [1,2,3]. This fact has limited broad application of gradient-based optimization of tensor network states to more complex systems Alternative approaches, such as computing the gradient using numerical derivatives, have limited accuracy and efficiency; they only apply to cases with few variational parameters [36,37]. Differentiable programming provides an elegant and efficient solution to these problems by composing the whole tensor network program in a fully differentiable manner and optimizing them using automatic differentiation.In this paper, we present essential automatic differentiation techniques that compute (higher-order) derivatives of tensor network programs efficiently to numeric precision This progress opens the door to gradient-based (and even Hessianbased) optimization of tensor network states in a general setting. Our code implementation is publicly available (see Ref. [39])
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
