Chapter 15 - Tensor methods in deep learning

Abstract

Tensors are multidimensional arrays that can naturally represent data and mappings of multiple dimensions, playing a central role in modern deep learning. Indeed, the basic building blocks of deep neural networks, such as multichannel convolutional kernels and attention blocks, are essentially multidimensional mappings represented by tensors. In effect, tensor methods are increasingly finding significant applications in deep learning, including the design of memory- and computation-efficient network architectures, in improving robustness to random noise and adversarial attacks, and in the interpretation of deep generative models. At the same time, tensor methods play a crucial role in the emerging field of quantum machine learning. This chapter provides an overview of tensors and tensor methods in the context of deep learning.