Temporal winner-take-all networks: a time-based mechanism for fast selection in neural networks

Abstract

Winner-take-all (WTA) networks frequently appear in neural network models. They are primarily used for decision making and selection. As an alternative to the conventional activation-based winner-take-all mechanisms (AWTA), we present a time-based temporal-winner-take-all mechanism with O(n) space complexity and roughly O(log n) time complexity. The mechanism exploits systematic and stochastic differences between time delays within different units and connections. The TWTA and the AWTA networks are shown to be logically equivalent, but the TWTA mechanism may be more suitable than the latter for various selection tasks, especially the selection of an arbitrary unit from a set (e.g., as in unit recruitment). TWTA avoids various problems with conventional WTA, notably the difficulty of making it converge rapidly over a large range of conditions. Here we report a probabilistic analysis of the TWTA mechanism along with experimental data obtained from numerous massively parallel simulations of the TWTA mechanism on the connection machine.