The Super Neuron Model - A new generation of ANN-based Machine Learning and Applications

Abstract

Operational Neural Networks (ONNs) are new generation network models targeting to address two major drawbacks of conventional Convolutional Neural Networks (CNNs): the homogenous network configuration and the “linear” neuron model that can only perform linear transformations over previous layer outputs. ONNs can perform any linear or non-linear transformation with a proper combination of “nodal” and “pool” operators. This is a great leap towards expanding the neuron’s learning capacity in CNNs, which thus far required the use of a single nodal operator for all synaptic connections for each neuron. This restriction has recently been lifted by introducing a superior neuron called the “generative neuron” where each nodal operator can be customized during the training in order to maximize learning. As a result, the network is able to self-organize the nodal operators of its neurons’ connections. Self-Organized ONNs (Self-ONNs) equipped with superior generative neurons can achieve diversity even with a compact configuration. We shall explore several signal processing applications of neural network models equipped with the superior neuron.