Spectral complexity-scaled generalisation bound of complex-valued neural networks

Abstract

Complex-valued neural networks (CVNNs) have been widely applied in various fields, primarily in signal processing and image recognition. Few studies have focused on the generalisation of CVNNs, although it is vital to ensure the performance of CVNNs on unseen data. This study is the first to prove a generalisation bound for complex-valued neural networks. The bounds increase as the spectral complexity increases, with the dominant factor being the product of the spectral norms of the weight matrices. Furthermore, this work provides a generalisation bound for CVNNs trained on sequential data, which is also affected by the spectral complexity. Theoretically, these bounds are derived using the Maurey Sparsification Lemma and Dudley entropy integral. We conducted empirical experiments on various datasets including MNIST, ashionMNIST, CIFAR-10, CIFAR-100, Tiny ImageNet, and IMDB by training complex-valued convolutional neural networks. The Spearman rank-order correlation coefficient and the corresponding p-values on these datasets provide strong proof of the statistically significant correlation between the spectral complexity of a network and its generalisation ability, as measured by the spectral norm product of the weight matrices. The code is available at https://github.com/LeavesLei/cvnn_generalization.