Winner-take-all Research Articles

PERFORMANCE EVALUATION OF FUZZY SINGLE LAYER WEIGHTLESS NEURAL NETWORK

The paper evaluates the performance of a neuro-fuzzy pattern classification system based on the weightless neural network architecture. The system utilizes a Single Layer Weightless Neural Network (SLWNN) to extract the features vector that measures the similarity of the input pattern to the different classification groups. In contrast to the traditional crisp Winner-Takes-All (WTA) classification scheme used by SLWNN, our system uses a Fuzzy Inference System (FIS) for classification. The network is trained by a hybrid learning scheme that combines a single pass learning phase for training the SLWNN followed by a supervised learning phase for extracting a set of fuzzy rules suitable to classify the training set. The FIS learns fuzzy rules from the feature vectors generated by the SLWNN for the set of training patterns. The recognition of handwritten numerals is employed as a test-bed to demonstrate the effectiveness of the proposed neuro-fuzzy system. Experimental results show that the performance of the proposed system surpasses the performance of the traditional SLWNN.

Training Winner-Take-All Simultaneous Recurrent Neural Networks

The winner-take-all (WTA) network is useful in database management, very large scale integration (VLSI) design, and digital processing. The synthesis procedure of WTA on single-layer fully connected architecture with sigmoid transfer function is still not fully explored. We discuss the use of simultaneous recurrent networks (SRNs) trained by Kalman filter algorithms for the task of finding the maximum among N numbers. The simulation demonstrates the effectiveness of our training approach under conditions of a shared-weight SRN architecture. A more general SRN also succeeds in solving a real classification application on car engine data.

Image segmentation utilizing the winner-take-all dynamics in a large-array opto-electronic feedback circuit

Image segmentation is a key element in processing image data. Done properly, image segmentation can both enhance the quality of subsequent processing and enable other higher-level processing to generate useful information. Image segmentation that clearly separates objects from the background can improve the accuracy of the recognition of the object. Isolating important objects within an image requires techniques that divide pixels into object or background information.A technique adopted from the neural network field is presented for performing image segmentation based on the winner-take-all (WTA) scheme implemented with an optoelectronic architecture. This combination allows the parallel nature of optics and the computational strengths of electronics to model a fast and efficient image segmentation system. A model of an architecture for a large array of optoelectronic feedback circuits that can be realized using new technology to perform image segmentation using the WTA scheme is proposed. The architecture is modeled optoelectronically as an interferometer with a simple nonlinearity for the control unit. Results from numerical analysis and simulations show the model can generate WTA behavior. Results from simulations are shown with sample images used to test the model's ability to perform segmentation.

Anticipation-Based Temporal Sequences Learning in Hierarchical Structure

Temporal sequence learning is one of the most critical components for human intelligence. In this paper, a novel hierarchical structure for complex temporal sequence learning is proposed. Hierarchical organization, a prediction mechanism, and one-shot learning characterize the model. In the lowest level of the hierarchy, we use a modified Hebbian learning mechanism for pattern recognition. Our model employs both active 0 and active 1 sensory inputs. A winner-take-all (WTA) mechanism is used to select active neurons that become the input for sequence learning at higher hierarchical levels. Prediction is an essential element of our temporal sequence learning model. By correct prediction, the machine indicates it knows the current sequence and does not require additional learning. When the prediction is incorrect, one-shot learning is executed and the machine learns the new input sequence as soon as the sequence is completed. A four-level hierarchical structure that isolates letters, words, sentences, and strophes is used in this paper to illustrate the model.

Using TD learning to simulate working memory performance in a model of the prefrontal cortex and basal ganglia

Delayed-response tasks (DRTs) have been used to assess working memory (WM) processes in human and nonhuman animals. Experiments have shown that the basal ganglia (BG) and dorsolateral prefrontal cortex (DLPFC) subserve DRT performance. Here, we report the results of simulation studies of a systems-level model of DRT performance. The model was trained using the temporal difference (TD) algorithm and uses an actor-critic architecture. The matrisomes of the BG represent the actor and the striosomes represent the critic. Unlike existing models, we hypothesize that the BG subserve the selection of both motor- and cognitive-related information in these tasks. We also assume that the learning of both processes is based on reward presentation. A novel feature of the model is the incorporation of delay-active neurons in the matrisomes, in addition to DLPFC. Another novel feature of the model is the subdivision of the matrisomal neurons into segregated winner-take-all (WTA) networks consisting of delay- versus transiently-active units. Our simulation model proposes a new neural mechanism to account for the occurrence of perseverative responses in WM tasks in striatal-, as well as in prefrontal damaged subjects. Simulation results also show that the model both accounts for the phenomenon of time shifting of dopamine phasic signals and the effects of partial reinforcement and reward magnitude on WM performance at both behavioral and neural levels. Our simulation results also found that the TD algorithm can subserve learning in delayed-reversal tasks.

A pixel array PSD with divide-by-M winner-take-all architecture

This paper presents a pixel array position sensitive detector (PSD) with analog winner-take-all (WTA) blocks. The pixels are divided into M-number of sub-groups and a WTA is assigned to each sub-group. The winner pixel of entire pixels is obtained from the sum of the sub-group's winner addresses. Detailed noise analysis shows that there is an optimal number of sub-groups for improved noise immunity.

A Compact Implementation Scheme of 1-Dimensional PSDs with Double-Resolution Interpolation

This paper proposes a compact interpolation scheme dedicated to a 1-dimensional position sensitive detector (PSD) with an optical sensing pixel array. The pixels are divided into even- and odd-numbered groups and winner take all (WTA) circuits are provided to each of the groups. The simulated results show that the detecting step-width is reduced to the half of the original one after applying the interpolation scheme.

The Scaling of Winner-Takes-All Accuracy with Population Size

Empirical studies seem to support conflicting hypotheses with regard to the nature of the neural code. While some studies highlight the role of a distributed population code, others emphasize the possibility of a "single-best-cell" readout. One particularly interesting example of single-best-cell readout is provided by the winner-takes-all (WTA) approach. According to the WTA, every cell is characterized by one particular preferred stimulus, to which it responds maximally. The WTA estimate for the stimulus is defined as the preferred stimulus of the cell with the strongest response. From a theoretical point of view, not much is known about the efficiency of single-best-cell readout mechanisms, in contrast to the considerable existing theoretical knowledge on the efficiency of distributed population codes. In this work, we provide a basic theoretical framework for investigating single-best-cell readout mechanisms. We study the accuracy of the WTA readout. In particular, we are interested in how the WTA accuracy scales with the number of cells in the population. Using this framework, we show that for large neuronal populations, the WTA accuracy is dominated by the tail of the single-cell-response distribution. Furthermore, we find that although the WTA accuracy does improve when larger populations are considered, this improvement is extremely weak compared to other types of population codes. More precisely, we show that while the accuracy of a linear readout scales linearly with the population size, the accuracy of the WTA readout scales logarithmically with the number of cells in the population.

Stochastic saliency-based search model for search asymmetry with uncertain targets

Whereas some models with mechanism coding task-related information, e.g. task-related saliency map, can explain why search asymmetry occurs in some visual search tasks where the target is preliminarily defined, existing models cannot clearly explain human behavior in singleton search tasks showing search asymmetry. This study shows that the saliency-based search model can explain search asymmetry in singleton search without assuming task-related saliency map if we introduce two new essential ideas. The first improvement is about a competitive mechanism between feature maps, and the other is the introduction of a stochastic winner takes all (WTA) network. According to the modified model introducing these two modifications, search asymmetry in singleton search tasks results from an asymmetry in probability of the attentional focus being directed to the target. When the saliency difference is large, then attentional focus is almost always directed to the target first. On the other hand, when the saliency difference is relatively small, then the probability of the target receiving attentional focus first becomes relatively smaller. The stochastic saliency-based search model could reproduce human behavior in a range of visual search tasks showing search asymmetry, without assuming task-related saliency map.

Learning vector quantization: The dynamics of winner-takes-all algorithms

Winner–Takes–All (WTA) prescriptions for learning vector quantization (LVQ) are studied in the framework of a model situation: two competing prototype vectors are updated according to a sequence of example data drawn from a mixture of Gaussians. The theory of on-line learning allows for an exact mathematical description of the training dynamics, even if an underlying cost function cannot be identified. We compare the typical behavior of several WTA schemes including basic LVQ and unsupervised vector quantization. The focus is on the learning curves, i.e. the achievable generalization ability as a function of the number of training examples.

Low power WTA circuit for optical position detector

A time domain winner take all (WTA) circuit for a 1D optical position detector is presented. The circuit implements a new asynchronous technique which estimates the position of a light spot impinging on the sensor with sub-pixel resolution. This technique first detects the peak of the light spot, then, starts a new competition, by eliminating the winner pixel. In this way the sensor asynchronously dispatches the pixels in order to reconstruct the spot profile. The simulated circuit shows a total power consumption of about 15.6 µW at 3.3 V, while the average dynamic power consumption needed to implement one step of the algorithm, can be estimated to be about 0.4 nW.

Condition monitoring of 3G cellular networks through competitive neural models

We develop an unsupervised approach to condition monitoring of cellular networks using competitive neural algorithms. Training is carried out with state vectors representing the normal functioning of a simulated CDMA2000 network. Once training is completed, global and local normality profiles (NPs) are built from the distribution of quantization errors of the training state vectors and their components, respectively. The global NP is used to evaluate the overall condition of the cellular system. If abnormal behavior is detected, local NPs are used in a component-wise fashion to find abnormal state variables. Anomaly detection tests are performed via percentile-based confidence intervals computed over the global and local NPs. We compared the performance of four competitive algorithms [winner-take-all (WTA), frequency-sensitive competitive learning (FSCL), self-organizing map (SOM), and neural-gas algorithm (NGA)] and the results suggest that the joint use of global and local NPs is more efficient and more robust than current single-threshold methods.

Low-voltage high-performance voltage-mode and current-mode WTA circuits based on flipped voltage followers

A new low-voltage CMOS winner-take-all (WTA) circuit is presented. The proposed circuit exhibits a linear increase of complexity with the number of inputs at the rate of only three transistors per input and it is based on a modified version of the common source scheme. In this case, each input follower is enhanced by local shunt feedback to increase its gain and to reduce its output impedance. Simulations demonstrate the potential of the circuit to operate at very high speed, with high precision and with a supply voltage close to a transistor's threshold voltage. Experimental verification of the circuit using a 0.5-/spl mu/m CMOS technology is also provided.

A winner-take-all circuit using neural networks as building blocks

A new analog inhibitory method based winner-take-all (WTA) circuit is proposed. The circuit structure is a binary tree arrangement of two-dimensional neural networks and logic nodes. The connection matrix belongs to the class of diagonally stable matrices and the activation functions are piecewise linear or sigmoidal. The mathematical justification of the strong WTA network functioning, on the basis of the uniqueness and global asymptotic stability of the equilibrium, is given. Simulation examples and the corresponding results are also provided.

A Columnar Competitive Model for Solving Combinatorial Optimization Problems

The major drawbacks of the Hopfield network when it is applied to some combinatorial problems, e.g., the traveling salesman problem (TSP), are invalidity of the obtained solutions, trial-and-error setting value process of the network parameters and low-computation efficiency. This letter presents a columnar competitive model (CCM) which incorporates winner-takes-all (WTA) learning rule for solving the TSP. Theoretical analysis for the convergence of the CCM shows that the competitive computational neural network guarantees the convergence to valid states and avoids the onerous procedures of determining the penalty parameters. In addition, its intrinsic competitive learning mechanism enables a fast and effective evolving of the network. The simulation results illustrate that the competitive model offers more and better valid solutions as compared to the original Hopfield network.

Single-electron winner-take-all network

A winner-take-all (WTA) single-electron neuron is developed for the first time. This new single-electron circuit is proposed in order to implement a WTA neural network with lateral inhibition architecture. An expression for the neuron's activation function is presented. Furthermore, a dot pattern recognition task is successfully performed by the implemented network considering effects such as offset charges and co-tunnelling.

Study of distributed learning as a solution to category proliferation in Fuzzy ARTMAP based neural systems

An evaluation of distributed learning as a means to attenuate the category proliferation problem in Fuzzy ARTMAP based neural systems is carried out, from both qualitative and quantitative points of view. The study involves two original winner-take-all (WTA) architectures, Fuzzy ARTMAP and FasArt, and their distributed versions, dARTMAP and dFasArt. A qualitative analysis of the distributed learning properties of dARTMAP is made, focusing on the new elements introduced to endow Fuzzy ARTMAP with distributed learning. In addition, a quantitative study on a selected set of classification problems points out that problems have to present certain features in their output classes in order to noticeably reduce the number of recruited categories and achieve an acceptable classification accuracy. As part of this analysis, distributed learning was successfully adapted to a member of the Fuzzy ARTMAP family, FasArt, and similar procedures can be used to extend distributed learning capabilities to other Fuzzy ARTMAP based systems.

A fast maxnet

Winner‐Take‐All (WTA) networks frequently appear in neural networks. They are primarily used for decision making and selection. Maxnet is a WTA network; it can select the maximum from a data set. However, there are two essential drawbacks to the Maxnet. The first drawback is its slow convergent speed if the input data have almost the same values or the amount of input data is very large. The second one is that it fails when nonunique extreme values exist. In this paper, a new WTA network model called Fast Maxnet is proposed to select the maximum from a data set and be able to converge to the correct state in any situation. Simulation results indicate that the fast Maxnet converges much faster than the Maxnet.

An APS with 2-D winner-take-all selection employing adaptive spatial filtering and false alarm reduction

An active pixel sensor (APS) with two-dimensional winner-take-all (WTA) detection is presented. This system-on-a-chip employs adaptive spatial filtering of the processed image, with bad pixel elimination and false alarm reduction in case of a missing object. The circuit has a unique ability of adaptive spatial filtering that allows removal of the background from the image, one stage before it is transferred to the WTA detection circuit. A test chip of a 64/sup */64 array has been implemented in 0.5-/spl mu/m CMOS technology, has a 49% fill factor, is operated by a 3.3-V supply, and dissipates 36 mW at video rate. System architecture and operation are discussed and measurements from the prototype chip are presented.

All fiber-optic neural network using coupled SOA based ring lasers

An all-optical neural network is presented that is based on coupled lasers. Each laser in the network lases at a distinct wavelength, representing one neuron. The network status is determined by the wavelength of the network's light output. Inputs to the network are in the optical power domain. The nonlinear threshold function required for neural-network operation is achieved optically by interaction between the lasers. The behavior of the coupled lasers is explained by a simple laser model developed in the paper. In particular, the winner take all (WTA) neural-network behavior of a system of many lasers is described. An experimental system is implemented using single mode fiber optic components at wavelengths near 1550 nm. A number of functions are implemented to demonstrate the practicality of the new network. The neural network is particularly robust against input wavelength variations.