Multi-scale tensor network architecture for machine learning

Abstract

We present an algorithm for supervised learning using tensor networks, employing a step of data pre-processing by coarse-graining through a sequence of wavelet transformations. These transformations are represented as a set of tensor network layers identical to those in a multi-scale entanglement renormalization ansatz tensor network. We perform supervised learning and regression tasks through a model based on a matrix product states (MPSs) acting on the coarse-grained data. Because the entire model consists of tensor contractions (apart from the initial non-linear feature map), we can adaptively fine-grain the optimized MPS model ‘backwards’ through the layers with essentially no loss in performance. The MPS itself is trained using an adaptive algorithm based on the density matrix renormalization group algorithm. We test our methods by performing a classification task on audio data and a regression task on temperature time-series data, studying the dependence of training accuracy on the number of coarse-graining layers and showing how fine-graining through the network may be used to initialize models which access finer-scale features.