Quadratic function nodes: Use, structure and training

Abstract

When computing on continuously valued features, quadratic function nodes provide a capability for prototypic categorization which linear nodes provide for binary features. This paper extends the formal analysis and empirical measurement of thresholded linear functions of multivalued features to determine the corresponding properties for quadratic functions. As with linear functions, the number of functions, weight size, training speed and number of nodes necessary to represent arbitrary Boolean functions are shown to increase polynomially with the number of distinct, equally-spaced values the input features can assume (that is with the required resolution), and exponentially with the number of features. Also paralleling linear functions, certain interesting subclasses of quadratic functions are shown to be learnable in polynomial time. These include functions which require exponential resources (nodes and training time) if a disjunction or conjunction of linear functions (a convex polygon) is used. Alternatively, if a distributed representation is possible, these functions can be represented with a linear number of linear nodes.