Plausible Algorithm Research Articles

Symbolic neural networks for cognitive capacities

Abstract Pattern recognition (recognizing a pattern from inputs) and recall (describing or predicting the inputs associated with a recognizable pattern) are essential for neural-symbolic processing and cognitive capacities. Without them the brain cannot interact with the world e.g.: form internal representations and recall memory upon which it can perform logic and reason. Neural networks are efficient, biologically plausible algorithms that can perform large scale recognition. However, most neural network models of recognition perform recognition but not recall: they are sub-symbolic. It remains difficult to connect models of recognition with models of logic and emulate fundamental brain functions, because of the symbolic recall limitation. I introduce a completely symbolic neural network that is similar in function to standard feedforward neural networks but uses feedforward-feedback connections similar to auto-associative networks. However, unlike auto-associative networks, the symmetrical feedback connections are inhibitory not excitatory. Initially it may seem counterintuitive that recognition can even occur because the top-down connections are self-inhibitory. The self-inhibitory configuration is used to implement a gradient descent mechanism that functions during recognition not learning. The purpose of the gradient-descent is not to learn weights (weights are still learned during learning) but to find neuron activation. The advantage of this approach is the weights can now be symbolic (representing prototypes of expected patterns) allowing recall within neural networks. Moreover, considering the costs of both learning and recognition, this approach may be more efficient than feedforward recognition. I show that this model mathematically emulates standard feedforward model equations in single layer networks without hidden units. Comparisons that include more layers are planned in the future.

Hebbian-based mean shift for learning the diverse shapes of V1 simple cell receptive fields

The L0-norm constraint in sparse coding has the advantage of producing the same diversity of receptive field shapes as physiology data, but is difficult for analysis. It remains a challenging issue to understand how the diverse shapes of V1 simple cell receptive fields emerge in visual cortex. This paper presents a biologically plausible learning algorithm, named Hebbian-based mean shift, for this problem. The L0-norm constraint optimizes the number of basis functions rather than their coefficients. We report that the optimization procedure is essentially a 0–1 programming of the selection of basis functions. By assuming that the basis functions are independently selected from a basis set, we find the spatial distribution of input samples containing a special basis function has a star shape and peaks at this basis function. Thus, learning the basis functions for sparse coding with the L0-norm can be interpreted as mode detection where the basis functions are the modes of the kernel density estimate. We employ mean shift to detect modes and prove that the updating rule for the mean shift is Hebbian. The simulation results demonstrate the robustness of the proposed algorithm in producing both Gabor-like and blob-like basis functions.

Classification using sparse representations: a biologically plausible approach

Representing signals as linear combinations of basis vectors sparsely selected from an overcomplete dictionary has proven to be advantageous for many applications in pattern recognition, machine learning, signal processing, and computer vision. While this approach was originally inspired by insights into cortical information processing, biologically plausible approaches have been limited to exploring the functionality of early sensory processing in the brain, while more practical applications have employed non-biologically plausible sparse coding algorithms. Here, a biologically plausible algorithm is proposed that can be applied to practical problems. This algorithm is evaluated using standard benchmark tasks in the domain of pattern classification, and its performance is compared to a wide range of alternative algorithms that are widely used in signal and image processing. The results show that for the classification tasks performed here, the proposed method is competitive with the best of the alternative algorithms that have been evaluated. This demonstrates that classification using sparse representations can be performed in a neurally plausible manner, and hence, that this mechanism of classification might be exploited by the brain.

Adaptive properties of differential learning rates for positive and negative outcomes

The concept of the reward prediction error-the difference between reward obtained and reward predicted-continues to be a focal point for much theoretical and experimental work in psychology, cognitive science, and neuroscience. Models that rely on reward prediction errors typically assume a single learning rate for positive and negative prediction errors. However, behavioral data indicate that better-than-expected and worse-than-expected outcomes often do not have symmetric impacts on learning and decision-making. Furthermore, distinct circuits within cortico-striatal loops appear to support learning from positive and negative prediction errors, respectively. Such differential learning rates would be expected to lead to biased reward predictions and therefore suboptimal choice performance. Contrary to this intuition, we show that on static "bandit" choice tasks, differential learning rates can be adaptive. This occurs because asymmetric learning enables a better separation of learned reward probabilities. We show analytically how the optimal learning rate asymmetry depends on the reward distribution and implement a biologically plausible algorithm that adapts the balance of positive and negative learning rates from experience. These results suggest specific adaptive advantages for separate, differential learning rates in simple reinforcement learning settings and provide a novel, normative perspective on the interpretation of associated neural data.

Electric Vehicles and the Smart Grid: Spatial Modelling of Impacts and Opportunities

In this paper we present a novel composite methodology for obtaining spatial projections of the impacts and opportunities arising from the integration of plug-in electric vehicles with future smart electricity grids. We link models of future plug-in electric vehicle uptake, travel by household members, household electricity demand, and recharge of electric vehicles. The analysis is disaggregated in each case to a mesh block or local government area level; vehicle usage and household energy demand fluctuate on a hourly, daily and seasonal basis, subject also to the longer-term trends projected for uptake of the new technology. A similarly fine grain is applied with respect to socio-economic variables. The uptake model combines features of choice modelling, multi-criteria analysis and technology diffusion theory; in this case it was applied to four competing technologies (BEV, PHEV, HEV, ICE), and calibration revealed seven major determinants of uptake: performance, annual costs, purchase cost, household income, driving distance, demographic suitability, and risk or inconvenience. The travel model projects likely patterns of vehicle usage and travel duration based on existing patterns of private vehicle usage. The household demand model includes detailed representation of housing type and usage of electrical appliances. The charge-discharge model embodies plausible algorithms for managing household electricity usage in conjunction with electric vehicle batteries. In the paper we describe the various models and report projected impacts of electric vehicles on peak electrical grid loads for the Australian state of Victoria. The impacts are presented on a spatial basis, to the level of individual mesh blocks and network feeders, under a range of energy management scenarios.

Haptic Interaction with Elastic Volumetric Structures

Considerable efforts have been done to produce realistic results when simulating interaction with elastic materials. Many applications such as surgery planning, medical training, or virtual sculpting would benefit from a plausible simulation scenario. However, even though many works have proposed very satisfactory results, realistic simulation of deformable bodies is still an open issue. One of the challenges when designing a realistic elastic body simulation is the huge amount of data that needs to be processed. For the inner properties of the material are crucial when it comes to reproduce the elastic problem, the simulation naturally calls for volumetric information. In this paper the authors propose a technique to interactively deform 3D images, such as those acquired by a CT scanner. While producing a physically plausible haptic feedback, deformation and visualization algorithms produce an efficient and natural feeling. Using a free form deformation structure as a wrapper, it is possible to deform complex structures at high frame rates, independently of the size of the volume.

Sparse and silent coding in neural circuits

Sparse coding algorithms find a linear basis in which signals can be represented by a small number of non-zero coefficients. Such coding may play an important role in neural information processing and metabolically efficient natural solutions serve as an inspiration for algorithms employed in various areas of computer science. In particular, finding non-zero coefficients in overcomplete sparse coding is a computationally hard problem, for which different approximate solutions have been proposed. Methods that minimize the magnitude of the coefficients (‘ ℓ 1 - norm ’) instead of minimizing the size of the active subset of features (‘ ℓ 0 - norm ’) may find the optimal solutions, but they do not scale well with the problem size and use centralized algorithms. Iterative, greedy methods, on the other hand are fast, but require a priori knowledge of the number of non-zero features, often find suboptimal solutions and they converge to the final sparse form through a series of non-sparse representations. In this article we propose a neurally plausible algorithm which efficiently integrates an ℓ 0 - norm based probabilistic sparse coding model with ideas inspired by novel iterative solutions. Furthermore, the resulting algorithm does not require an exactly defined sparseness level thus it is suitable for representing natural stimuli with a varying number of features. We demonstrate that our combined method can find optimal solutions in cases where other, ℓ 1 - norm based algorithms already fail.

Unmixing binocular signals

Incompatible images presented to the two eyes lead to perceptual oscillations in which one image at a time is visible. Early models portrayed this binocular rivalry as involving reciprocal inhibition between monocular representations of images, occurring at an early visual stage prior to binocular mixing. However, psychophysical experiments found conditions where rivalry could also occur at a higher, more abstract level of representation. In those cases, the rivalry was between image representations dissociated from eye-of-origin information, rather than between monocular representations from the two eyes. Moreover, neurophysiological recordings found the strongest rivalry correlate in inferotemporal cortex, a high-level, predominantly binocular visual area involved in object recognition, rather than early visual structures. An unresolved issue is how can the separate identities of the two images be maintained after binocular mixing in order for rivalry to be possible at higher levels? Here we demonstrate that after the two images are mixed, they can be unmixed at any subsequent stage using a physiologically plausible non-linear signal-processing algorithm, non-negative matrix factorization, previously proposed for parsing object parts during object recognition. The possibility that unmixed left and right images can be regenerated at late stages within the visual system provides a mechanism for creating various binocular representations and interactions de novo in different cortical areas for different purposes, rather than inheriting then from early areas. This is a clear example how non-linear algorithms can lead to highly non-intuitive behavior in neural information processing.

Compressed sensing with stochastic spikes

Compressed Sensing (CS) refers to the mathematical finding that perfect reconstruction of a high dimensional state can be possible also from much lower dimensional samples provided the state representation is sufficiently sparse. Since neuronal activities in cortex are in fact sparse it is tempting to explain certain aspects of neuronal coding in terms of CS ([1] and [2]). The applicability of CS to neuronal structures and activities, however, critically relies on realistic assumptions about the neuronal mechanisms that could implement efficient algorithms. Here we investigated the feasibility of CS with rate coding neurons. Also, we are interested in the speed-precision tradeoff of reconstructions using spike-based algorithms similar to the one we introduced previously [3]. We find that a biologically plausible algorithm for non-negative activities can efficiently exploit the information contained in stochastic spike events and converges to close solutions for a wide range of sparsenesses and under-samplings. Figure ​Figure11 shows the root mean squared error (relative to the initial error) averaged over 10 examples depending on proportion of zeros in the representation on the abscissa and the compression ratio M/N on the ordinate, where M is number of observed neurons and N (here set to 500) is the dimension of the underlying state. Figure 1 We also investigated conditions on the generating matrix that would facilitate satisfactory reconstructions from limited numbers of spikes. Learning such structures with sparseness constraints can speed up estimations but will in general not match the 'true' generating model. Therefore the construction of sparse representations from spikes can be considered a bias favoring speed in contrast to faithfulness. In [3] we showed that learning generating matrices is possible using only spike activity. Taken together, our results underline the potential relevance of CS for understanding connectivity structures, sparseness and activity dynamics in the brain.

Online unsupervised formation of cell assemblies for the encoding of multiple cognitive maps

Since their introduction sixty years ago, cell assemblies have proved to be a powerful paradigm for brain information processing. After their introduction in artificial intelligence, cell assemblies became commonly used in computational neuroscience as a neural substrate for content addressable memories. However, the mechanisms underlying their formation are poorly understood and, so far, there is no biologically plausible algorithms which can explain how external stimuli can be online stored in cell assemblies. We addressed this question in a previous paper [Salihoglu, U., Bersini, H., Yamaguchi, Y., Molter, C., (2009). A model for the cognitive map formation: Application of the retroaxonal theory. In Proc. IEEE international joint conference on neural networks], were, based on biologically plausible mechanisms, a novel unsupervised algorithm for online cell assemblies' creation was developed. The procedure involved simultaneously, a fast Hebbian/anti-Hebbian learning of the network's recurrent connections for the creation of new cell assemblies, and a slower feedback signal which stabilized the cell assemblies by learning the feedforward input connections. Here, we first quantify the role played by the retroaxonal feedback mechanism. Then, we show how multiple cognitive maps, composed by a set of orthogonal input stimuli, can be encoded in the network. As a result, when facing a previously learned input, the system is able to retrieve the cognitive map it belongs to. As a consequence, ambiguous inputs which could belong to multiple cognitive maps can be disambiguated by the knowledge of the context, i.e. the cognitive map.

On the analysis of glycomics mass spectrometry data via the regularized area under the ROC curve

BackgroundNovel molecular and statistical methods are in rising demand for disease diagnosis and prognosis with the help of recent advanced biotechnology. High-resolution mass spectrometry (MS) is one of those biotechnologies that are highly promising to improve health outcome. Previous literatures have identified some proteomics biomarkers that can distinguish healthy patients from cancer patients using MS data. In this paper, an MS study is demonstrated which uses glycomics to identify ovarian cancer. Glycomics is the study of glycans and glycoproteins. The glycans on the proteins may deviate between a cancer cell and a normal cell and may be visible in the blood. High-resolution MS has been applied to measure relative abundances of potential glycan biomarkers in human serum. Multiple potential glycan biomarkers are measured in MS spectra. With the objection of maximizing the empirical area under the ROC curve (AUC), an analysis method was considered which combines potential glycan biomarkers for the diagnosis of cancer.ResultsMaximizing the empirical AUC of glycomics MS data is a large-dimensional optimization problem. The technical difficulty is that the empirical AUC function is not continuous. Instead, it is in fact an empirical 0–1 loss function with a large number of linear predictors. An approach was investigated that regularizes the area under the ROC curve while replacing the 0–1 loss function with a smooth surrogate function. The constrained threshold gradient descent regularization algorithm was applied, where the regularization parameters were chosen by the cross-validation method, and the confidence intervals of the regression parameters were estimated by the bootstrap method. The method is called TGDR-AUC algorithm. The properties of the approach were studied through a numerical simulation study, which incorporates the positive values of mass spectrometry data with the correlations between measurements within person. The simulation proved asymptotic properties that estimated AUC approaches the true AUC. Finally, mass spectrometry data of serum glycan for ovarian cancer diagnosis was analyzed. The optimal combination based on TGDR-AUC algorithm yields plausible result and the detected biomarkers are confirmed based on biological evidence.ConclusionThe TGDR-AUC algorithm relaxes the normality and independence assumptions from previous literatures. In addition to its flexibility and easy interpretability, the algorithm yields good performance in combining potential biomarkers and is computationally feasible. Thus, the approach of TGDR-AUC is a plausible algorithm to classify disease status on the basis of multiple biomarkers.

Model of Frequency Analysis in the Visual Cortex and the Shape from Texture Problem

This paper addresses the question: at the level of cortical cells present in the primary area V1, is the information sufficient to extract the local perspective from the texture? Starting from a model of complex cells in visual area V1, we propose a biologically plausible algorithm for frequency analysis applied to the shape from texture problem. First, specific log-normal filters are designed in replacement of the classical Gabor filters because of their theoretical properties and of their biological plausibility. These filters are separable in frequency and orientation and they better sample the image spectrum which makes them appropriate for any pattern analysis technique. A method to estimate the local frequency in the image, which discards the need to choose the best local scale, is designed. Based on this frequency analysis model, a local decomposition of the image into patches leads to the estimation of the local frequency variation which is used to solve the problem of recovering the shape from the texture. From the analytical relation between the local frequency and the geometrical parameters, under perspective projection, it is possible to recover the orientation and the shape of the original image. The accuracy of the method is evaluated and discussed on different kind of textures, both regular and irregular, with planar and curved surfaces and also on natural scenes and psychophysical stimuli. It compares favorably to the best existing methods, with in addition, a low computational cost. The biological plausibility of the model is finally discussed.

Efficient Computation Based on Stochastic Spikes

The speed and reliability of mammalian perception indicate that cortical computations can rely on very few action potentials per involved neuron. Together with the stochasticity of single-spike events in cortex, this appears to imply that large populations of redundant neurons are needed for rapid computations with action potentials. Here we demonstrate that very fast and precise computations can be realized also in small networks of stochastically spiking neurons. We present a generative network model for which we derive biologically plausible algorithms that perform spike-by-spike updates of the neuron's internal states and adaptation of its synaptic weights from maximizing the likelihood of the observed spike patterns. Paradigmatic computational tasks demonstrate the online performance and learning efficiency of our framework. The potential relevance of our approach as a model for cortical computation is discussed.

A spike-based computational model for noise robust vowel classification

The human ability to recognize speech drastically outperforms that of commercial ASR systems especially in noisy environments. Presently, there is limited knowledge of the auditory system dynamics, however it is known that coding and processing of information is carried out via action potentials. This research aims to better understand the coding mechanisms along the auditory pathway, while devising a noise robust system for speech recognition. A biologically plausible algorithm for vowel classification is proposed, which solely uses spikes for both the feature extraction and the classification stages. The algorithm uses an improved and adaptive model of the inner-hair cell [Sumner et al., J. Acoust. Soc. Am. 113, 893–901 (2003)] to generate spike trains at different characteristic frequencies. The synchrony among the hair cells is used as a noise robust means for feature extraction. Detected features are then classified using a spike-based rank order coder, which uses the spike arrival times to the postsynaptic neuron to encode information [Delorme and Thorpe, Neural Networks. 14, 795–803 (2001)]. Experiments on a noisy vowel dataset (5 dB SNR) show an average of 15% increase in the recognition rate for the prototype system when compared to a nearest-neighbor classifier employing Mel frequency cepstral coefficients.

Sparse overcomplete Gabor wavelet representation based on local competitions

Gabor representations present a number of interesting properties despite the fact that the basis functions are nonorthogonal and provide an overcomplete representation or a nonexact reconstruction. Overcompleteness involves an expansion of the number of coefficients in the transform domain and induces a redundancy that can be further reduced through computational costly iterative algorithms like Matching Pursuit. Here, a biologically plausible algorithm based on competitions between neighboring coefficients is employed for adaptively representing any source image by a selected subset of Gabor functions. This scheme involves a sharper edge localization and a significant reduction of the information redundancy, while, at the same time, the reconstruction quality is preserved. The method is characterized by its biological plausibility and promising results, but it still requires a more in depth theoretical analysis for completing its validation.

Surface segmentation based on the luminance and color statistics of natural scenes.

The luminance and color of surfaces in natural scenes are relatively independent under certain linear transformations, with the luminance of a surface providing little information about the color of that surface, and vice versa. However, differences in luminance between two locations in a natural scene remain strongly associated with differences in color. We used the statistics of the spatiochromatic structure of natural scenes as the priors for a Bayesian model that decides whether or not two points within an image fall on the same surface. This model provides a biologically plausible algorithm for surface segmentation that models observer segmentations well.

A simple mechanism for curvature detection

This paper presents a simple and biologically plausible algorithm for detecting and discriminating between line features of various curvatures. The algorithm involves filtering the input image with a set of directional filters, and computing the filter response energy (local energy). The curvature information is obtained from analysing the local energy information. The algorithm is insensitive to noise (up to 64 dB) and the orientation of the features present in the input image.

Pattern analysis with spiking neurons using delay coding

Spiking neurons, receiving temporally encoded inputs, can compute radial basis functions (RBFs) by storing the relevant information in their delays. These delays can be learned using exclusively locally available information (basically the time difference between the pre- and post-synaptic spike). Our approach gives rise to a biologically plausible algorithm for finding clusters in a high-dimensional input space. Furthermore, we show that our learning mechanism makes it possible that such RBF neurons can perform some kind of feature extraction. Finally we demonstrate that this model allows the recognition of temporal sequences even if they are distorted in various ways.

Robust and optimal use of information in stereo vision.

Differences between the left and right eye's views of the world carry information about three-dimensional scene structure and about the position of the eyes in the head. The contemporary Bayesian approach to perception implies that human performance in using this source of eye-position information can be analysed most usefully by comparison with the performance of a statistically optimal observer. Here we argue that the comparison observer should also be statistically robust, and we find that this requirement leads to qualitatively new behaviours. For example, when presented with a class of stereoscopic stimuli containing inconsistent information about eccentricity of gaze, estimates of this gaze parameter recorded from one robust ideal observer bifurcate at a critical value of stimulus inconsistency. We report an experiment in which human observers also show this phenomenon and we use the experimentally determined critical value to estimate the vertical acuity of the visual system. The Bayesian analysis also provides a highly reliable and biologically plausible algorithm that can recover eye positions even before the classic stereo-correspondence problem is solved, that is, before deciding which features in the left and right images are to be matched.

Ethical coherence

This paper explores the ethical relevance of a precise new characterization of coherence as maximization of satisfaction of positive and negative constraints. A coherence problem can be stated by specifying a set of elements to be accepted or rejected along with sets of positive and negative constraints that incline pairs of elements to be accepted together or rejected together. Computationally tractable and psychologically plausible algorithms are available for determining the acceptance and rejection of elements in a way that reliably approximates coherence maximization. This paper shows how justification of ethical principles and particular judgments can be accomplished by taking into account deductive, explanatory, analogical, and deliberative coherence.