Matrix derivatives with chain rule and rules for simple, Hadamard, and Kronecker products

Abstract

The differential calculus for scalars is used to develop theorems for a calculus of functions of matrices. No appeal to scalar notation is necessary in the resulting calculus, so that the given chain and matrix product rules have wide applicability to matrix theory and models. The chain and product rules of scalar calculus become a special case of the matrix rules. The methods are illustrated by application to a generalized stochastic process model potentially relevant to long- and short-term memory.