A Block-Sparse Tensor Train Format for Sample-Efficient High-Dimensional Polynomial Regression

Michael Götte,Reinhold Schneider,Philipp Trunschke

doi:10.3389/fams.2021.702486

Michael Götte, Reinhold Schneider + Show 1 more

Open Access

https://doi.org/10.3389/fams.2021.702486

Copy DOI

Abstract

Low-rank tensors are an established framework for the parametrization of multivariate polynomials. We propose to extend this framework by including the concept of block-sparsity to efficiently parametrize homogeneous, multivariate polynomials with low-rank tensors. This provides a representation of general multivariate polynomials as a sum of homogeneous, multivariate polynomials, represented by block-sparse, low-rank tensors. We show that this sum can be concisely represented by a single block-sparse, low-rank tensor.We further prove cases, where low-rank tensors are particularly well suited by showing that for banded symmetric tensors of homogeneous polynomials the block sizes in the block-sparse multivariate polynomial space can be bounded independent of the number of variables.We showcase this format by applying it to high-dimensional least squares regression problems where it demonstrates improved computational resource utilization and sample efficiency.

Highlights

An important problem in many applications is the identification of a function from measurements or random samples
We propose to extend this framework by including the concept of blocksparsity to efficiently parametrize homogeneous, multivariate polynomials with low-rank tensors
We further prove cases, where low-rank tensors are well suited by showing that for banded symmetric tensors of homogeneous polynomials the block sizes in the blocksparse multivariate polynomial space can be bounded independent of the number of variables

Summary

Introduction

An important problem in many applications is the identification of a function from measurements or random samples. For this problem to be well-posed, some prior information about the function has to be assumed and a common requirement is that the function can be approximated in a finite dimensional ansatz space. The applicability of SINDy to highdimensional problems is limited since truly high-dimensional problems require a nonlinear parameterization of the ansatz space. One particular reparametrization that has proven itself in many applications are tensor networks These allow for a straight-forward extension of SINDy [2] but can encode additional structure as presented in [3]. In the context of optimal control tensor train networks have been utilized for solving the Hamilton–Jacobi–Bellman equation in [8,9], for solving backward stochastic differential equations in [10] and for the

Methods

Results

Conclusion