Large scale tensor regression using kernels and variational inference

Robert Hu,Geoff K Nicholls,Dino Sejdinovic

doi:10.1007/s10994-021-06067-7

Abstract

We outline an inherent flaw of tensor factorization models when latent factors are expressed as a function of side information and propose a novel method to mitigate this. We coin our methodology kernel fried tensor (KFT) and present it as a large-scale prediction and forecasting tool for high dimensional data. Our results show superior performance against LightGBM and Field aware factorization machines (FFM), two algorithms with proven track records, widely used in large-scale prediction. We also develop a variational inference framework for KFT which enables associating the predictions and forecasts with calibrated uncertainty estimates on several datasets.

Highlights

Introduction and related workIn recent times, industrial prediction problems (Caro and Gallien 2010; Seeger et al 2016) are large scale and high dimensional (Zhai et al 2014)
Tensor models are used in large-scale prediction tasks ranging from bioinformatics to industrial prediction; we work in the latter setting
While existing tensor models are versatile and scalable, they have a flaw: When we build a model of the latent factors in tensor factorization as a function of covariates (Agarwal and Chen 2009; Kim et al 2016; Kim and Choi 2014; Zhang et al 2014), the model may be restricted by global parameter couplings that are generated

Summary

Introduction

Industrial prediction problems (Caro and Gallien 2010; Seeger et al 2016) are large scale and high dimensional (Zhai et al 2014) Problems of this nature are ubiquitous and the most common setting is, but not limited to, the recommendation systems (Bobadilla et al 2013). While existing tensor models are versatile and scalable, they have a flaw: When we build a model of the latent factors in tensor factorization as a function of covariates (Agarwal and Chen 2009; Kim et al 2016; Kim and Choi 2014; Zhang et al 2014), the model may be restricted by global parameter couplings that are generated These couplings lead to a reduction of tractability at scale.

Objectives

Methods

Findings

Conclusion