Canonical piecewise-linear networks

Abstract

In this paper, mapping networks will be considered from the viewpoint of the piecewise-linear (PWL) approximation. The so-called canonical representation plays a kernel role in the PWL representation theory. While this theory has been researched intensively in the contents of mathematics and circuit simulations, little has been seen in the research area about the theoretical aspect of neural networks. This paper modifies this theory and applies it as a mathematical support for mapping networks. The main modification is a "higher-level" generalization of the canonical representation with proofs of its availability in the set of PWL functions. The modified theory will first be used to study the canonical PWL feature of the popular multilayer perceptron-like (MLPL) networks. Second, it will be seen that the generalized canonical representation is itself suitable for a network implementation, which is called the standard canonical PWL network. More generally, the family of (generalized) canonical PWL networks is defined as those which may take the canonical PWL representation as a mathematical model. This family is large and practically meaningful. The standard canonical PWL networks may be taken as representatives in the family. The modification of the PWL representation theory as well as the introduction of this theory in the theoretical study of mapping networks, which provide a new concept of mapping networks, i.e., the canonical PWL network family, may be regarded as the main contributions of the paper.