Perceptron Learning Rule Research Articles

Application of Improved Class of Weighted Averaging Operators in Intuitionistic Fuzzy Ann Guided MAGDM E-Commerce Problems

Objectives: To propose a novel and improvised Artificial Neural Network (ANN) technique to solve Multiple Attribute Group Decision Making (MAGDM) problems and an improved class of Aggregation operators for combining Intuitionistic Fuzzy Set (IFS) matrices for the ANN algorithm. Methods: A novel class of improved aggregation operators namely the Improved Intuitionistic Fuzzy Weighted Arithmetic Averaging (IM-IFWAA) operator and the Improved Intuitionistic Fuzzy Ordered Weighted Averaging (IM-IFOWA) operator are proposed in this work. The proposed improved class of aggregation operators will aggregate the IFS matrix data sets appearing in the form of matrices and then the revised input vectors which is then fed into the ANN algorithm following Delta, Perceptron, and Hebb Learning Rule for the next phase. Findings: Aggregating the Intuitionistic Fuzzy set information or data in today’s digital world is a tedious task in the field of MAGDM problem-solving. In this work, two new classes of operators for aggregating the data sets are proposed namely the IM-IFWAA operator and the IM-IFOWA operator which will improve the performance of the ANN. Necessary theorems for the proposed operators are proved to provide consistency of the same. The input created using the aforementioned operators is then fed into the novel ANN algorithm for further computations. Varieties of learning rules are then engaged for ranking of the best alternative for the decision problem. The developed theory is supported by a numerical example which is computed using all the proposed techniques. Novelty: Most of the research done on Intuitionistic Fuzzy Artificial Neural Network models are based on learning rules or using some other calculations. The proposed methods of using novel and improvised aggregation operators for ANN are used to find the inputs for ANN where varieties of learning rules for ANN are employed for effective decision analysis. Keywords: ANN, Learning Rules, MAGDM, Intuitionistic Fuzzy Sets, Weighted Aggregation Operator

Parçacık Sürü Optimizasyonu Yoluyla Geliştirilen Doğrusal Bir Sınıflandırıcının Analizi

Meta-heuristics are high-level approaches developed to discover a heuristic that provides a reasonable solution to many varieties of optimization problems. The classification problems contain a sort of optimization problem. Simply, the objective herein is to reduce the number of misclassified instances. In this paper, the question of whether meta-heuristic methods can be used to construct linear models or not is answered. To this end, Particle Swarm Optimization (PSO) has been engaged to address linear classification problems. The Particle Swarm Classifier (PSC) with a certain objective function has been compared with Support Vector Machine (SVM), Perceptron Learning Rule (PLR), and Logistic Regression (LR) applied to fifteen data sets. The experimental results point out that PSC can compete with the other classifiers, and it turns out to be superior to other classifiers for some binary classification problems. Furthermore, the average classification accuracies of PSC, SVM, LR, and PLR are 80.8%, 80.6%, 80.9%, and 57.7%, respectively. In order to enhance the classification performance of PSC, more advanced objective functions can be developed. Further, the classification accuracy can be boosted more by constructing tighter constraints via another meta-heuristic.

The third AI summer: AAAI Robert S. Engelmore Memorial Lecture

The third AI summer: AAAI Robert S. Engelmore Memorial Lecture

A Novel Approach for Data Classification Using Neural Networks

In this paper, an approach of multi-view learning, with multilayer perceptron (MLP) and radial basis functions (RBF) with dynamic decay adjustment (DDA), has been proposed. Three different categories of semi-supervised learning are multi-view training, co-training and self-training. Here we have only used self-training and multi-view learning mechanisms to train the classifier. To test the accuracy of the algorithms, we have taken five real-time datasets from UCI Machine Learning Repository. The classifier is trained using the perceptron learning rule with its supervised and semi-supervised (self-training) versions and MLP with RBF (multi-view learning). The average classification accuracies have been compared and the proposed algorithm outperforms the former versions on the specified training sets. The significant improvement in performance obtained using multi-view learning can be used for various fields such as detecting changes of images, speech recognition and biometric identification.

Dimensions reduction of vibration signal features using LDA and PCA for real time tool wear detection with single layer perceptron

This study uses the Linear Discriminant Analysis (LDA) method along with the Principal Component Analysis (PCA) method to reduce the dimensionality of the vibration signal feature classified by Single Layer Perceptron (SLP). The vibration features to be reduced are 10 out of 270 features selected based on the correlations analysis. The LDA and PCA transformations provide only three inputs, than the original 10 signal features for the SLP classifier. The Single Layer Perceptron is trained with a sequential incremental training approach using the perceptron learning rule. The training phase of SLP resulting Mean Squared Error (MSE) as low as 0.0840 and the validation phase in the CNC machining provides 97.5% accuracy with zero false alarms.

Alphanumeric Pattern Recognition by Memristive Crossbar Circuit using Perceptron Learning Rule

This paper presents an alphanumeric pattern recognition approach based on memristive crossbar circuit using perceptron learning rule. The proposed approach incorporates a memristive crossbar-based learning and training circuit (TC) module (i.e., synaptic network) and an operational amplifier (op-amp)-based neuron. Alphanumeric patterns, such as alphabets (A–Z) and numerics (0–9), are applied on the TC module and it adjusts the synaptic weights using the perceptron learning rule. The TC module includes 16 inputs, which are interconnected to nine output neurons through memristors. The input and output patterns are represented through [Formula: see text] and [Formula: see text] matrix pixels, respectively. This proposed circuit has implemented all alphanumeric patterns, such as alphabets (A–Z) and numerics (0–9), successfully. However, only the pattern “A” is illustrated in detail for better understanding. SPICE simulation results supported by analytical calculations of pattern “A” are reported. The average power consumption for the proposed approach using memristor is 77.77% lower than the conventional MOSFET-based approach, apart from significant saving of silicon overhead in contrast to its counterpart approach.

Improved Learning in Constructive Neural Networks for Pattern Classification using Genetic Algorithm

Objective: To remove rogue entries in the data set chose for training thus making the Neural Network (NN) learning easier. Methods/Analysis: In this research work Genetic Algorithm (GA) is being used on the data sets Wine, Pima Indians Diabetes, Iris, Vehicle and Image Segmentation from the UC Irvine machine learning repository. The refined data sets are then fed to a Constructive Neural Network (CoNN) for pattern classification task. CoNN dispenses an optimal strategy in designing the structures of the multilayer perceptron networks, coupled with supervised algorithms for further learning. They eliminate the requisite of ad-hoc, apriori design of network architecture which is often inappropriate. The M-Tiling algorithm is used for constructing the network and a simple perceptron learning rule for training individual TLUs (Threshold Logic Unit). The CoNNs potential capability is in constructing networks with a network size adequate to the complexity level of the task and in attaining a satisfactory level of efficiency. Findings: Acquiring data from real world problems may result in the frequent occurrence of rogue values. Rogue values result in poor data quality and mislead the learning curve yielding imprecise NN models. A template string prototype is generated with the continual application of GA on the input data. This template string prototype removes the rogue values from the actual training set which results in the reduction of the size of input data set, in turn, makes learning easier. CoNN coupled with GA classifies patterns in Wine, Pima Indians Diabetes, Iris, Vehicle and Image Segmentation datasets available in UC Irvine machine learning repository with faster convergence, more generalization accuracy, and less space. The proposed work is unique since GA is exerted to the training data set before it is fed to the CoNN for learning. The proposed work also demonstrates the improvement of CoNN in terms of convergence, generalization accuracy, and space optimization. In this way, these experimental results add value to the currently existing work. Improvements: The proposed model can be further revamped by using other CoNN learning strategies.

Cloud-assisted QoE guarantee scheme based on adaptive cross-layer perceptron of artificial neural network for mobile Internet

For improving the system performance of mobile Internet, how to provide the Quality of Experience (QoE) guarantee is an important factor. First, based on artificial neural network and adaptive cross-layer perceptron, we studied the cloud-assisted QoE guarantee mechanism. Then, according to the power, we divided the distance and perceptron layers of mobile Internet and cloud into three levels. We showed the state information definition of the mobile node on the basis of the adaptive adjustment perceptron layers. Thirdly, the perceptron network topology would be updated according to the customer service, which would be updated based on the perceptron learning rule for improving the training practice efficiency. The above scheme would guarantee the QoE effectively. The experimental results show that the proposed QoE guarantee mechanism has obvious advantages in terms of throughput, efficiency, and reliability.

Implementation of a spike-based perceptron learning rule using TiO2-x memristors.

Synaptic plasticity plays a crucial role in allowing neural networks to learn and adapt to various input environments. Neuromorphic systems need to implement plastic synapses to obtain basic “cognitive” capabilities such as learning. One promising and scalable approach for implementing neuromorphic synapses is to use nano-scale memristors as synaptic elements. In this paper we propose a hybrid CMOS-memristor system comprising CMOS neurons interconnected through TiO2−x memristors, and spike-based learning circuits that modulate the conductance of the memristive synapse elements according to a spike-based Perceptron plasticity rule. We highlight a number of advantages for using this spike-based plasticity rule as compared to other forms of spike timing dependent plasticity (STDP) rules. We provide experimental proof-of-concept results with two silicon neurons connected through a memristive synapse that show how the CMOS plasticity circuits can induce stable changes in memristor conductances, giving rise to increased synaptic strength after a potentiation episode and to decreased strength after a depression episode.

Special Issue on Autonomous Learning

ion of learning processes as observed in neurobiology; quite a few early learning models are based on biological paradigms such as Hebbian learning, i.e. the corresponding algorithms are based on heuristics—the perceptron learning rule for classification or Oja’s rule for dimensionality reduction constitute prominent examples [11]. Albeit these learning rules often reveal a striking simplicity, their mathematical foundation can be quite complicated, even partially unsolved [4, 9]. With time, more and more mathematical formalisms emerged within machine learning which enable researchers to derive a learning rule based on an explicit mathematical modelling of the underlying goal: most modern techniques are accompanied by mathematical cost functions such as the least squares error for regression, constraint margin maximisation for robust classification in support vector machines, or the data log likelihood for model inference [5]. While this mathematical treatment leads to highly efficient learning techniques, it often narrows the applicability of the techniques to specific areas which are covered by the considered cost function and data representation. This way, the machine learning algorithm is uncoupled from the question what is a good cost function and parameterisation of it, what is a good data representation, and what is an informative data set based on which learning becomes possible. These crucial ingredients of machine & Barbara Hammer bhammer@techfak.uni-bielefeld.de Marc Toussaint marc.toussaint@informatik.uni-stuttgart.de 1 CITEC Centre of Excellence, Bielefeld University, 33594 Bielefeld, Germany 2 Machine Learning and Robotics Lab, University of Stuttgart, 70569 Stuttgart, Germany 123 Kunstl Intell (2015) 29:323–327 DOI 10.1007/s13218-015-0392-x

A Three-Threshold Learning Rule Approaches the Maximal Capacity of Recurrent Neural Networks.

Understanding the theoretical foundations of how memories are encoded and retrieved in neural populations is a central challenge in neuroscience. A popular theoretical scenario for modeling memory function is the attractor neural network scenario, whose prototype is the Hopfield model. The model simplicity and the locality of the synaptic update rules come at the cost of a poor storage capacity, compared with the capacity achieved with perceptron learning algorithms. Here, by transforming the perceptron learning rule, we present an online learning rule for a recurrent neural network that achieves near-maximal storage capacity without an explicit supervisory error signal, relying only upon locally accessible information. The fully-connected network consists of excitatory binary neurons with plastic recurrent connections and non-plastic inhibitory feedback stabilizing the network dynamics; the memory patterns to be memorized are presented online as strong afferent currents, producing a bimodal distribution for the neuron synaptic inputs. Synapses corresponding to active inputs are modified as a function of the value of the local fields with respect to three thresholds. Above the highest threshold, and below the lowest threshold, no plasticity occurs. In between these two thresholds, potentiation/depression occurs when the local field is above/below an intermediate threshold. We simulated and analyzed a network of binary neurons implementing this rule and measured its storage capacity for different sizes of the basins of attraction. The storage capacity obtained through numerical simulations is shown to be close to the value predicted by analytical calculations. We also measured the dependence of capacity on the strength of external inputs. Finally, we quantified the statistics of the resulting synaptic connectivity matrix, and found that both the fraction of zero weight synapses and the degree of symmetry of the weight matrix increase with the number of stored patterns.

Normalization of input patterns in an associative network

Numerous brain structures have a cerebellum-like architecture in which inputs diverge onto a large number of granule cells that converge onto principal cells. Plasticity at granule cell-to-principal cell synapses is thought to allow these structures to associate spatially distributed patterns of granule cell activity with appropriate principal cell responses. Storing large sets of associations requires the patterns involved to be normalized, i.e., to have similar total amounts of granule cell activity. Using a general model of associative learning, we describe two ways in which granule cells can be configured to promote normalization. First, we show how heterogeneity in firing thresholds across granule cells can restrict pattern-to-pattern variation in total activity while also limiting spatial overlap between patterns. These effects combine to allow fast and flexible learning. Second, we show that the perceptron learning rule selectively silences those synapses that contribute most to pattern-to-pattern variation in the total input to a principal cell. This provides a simple functional interpretation for the experimental observation that many granule cell-to-Purkinje cell synapses in the cerebellum are silent. Since our model is quite general, these principles may apply to a wide range of associative circuits.

Pattern separation by neuronal turnover in a feed-forward network

Numerous studies have revealed a role for the hippocampal dentate gyrus in behavioral discrimination between similar contexts or objects, referred to as pattern separation. Recently many behavioral studies have demonstrated a role for dentate neurogenesis in such pattern separation. While several computational studies have modeled the effect of neuronal turnover on learning in simple [1-3] and more detailed networks [4], the computational advantage of neurogenesis for pattern separation has remained obscure. Here we present a simple, feed-forward network, with a biologically plausible learning rule for implementing neuronal recycling that reveals the pattern separation properties of neurogenesis. Our model consists of a three-layer network including entorhinal cortical inputs (EC), dentate gyrus (DG) as the hidden layer, and a single CA3 readout unit. The weights between EC and DG are fixed random, while those from DG to CA3 are trained with a perceptron learning rule. A context consists of a set of patterns presented to EC with their respective target readouts at CA3. Pereceptron training results in a weight vector with approximately normally distributed weights. Neurogenesis is implemented by replacing units with weak weights to the CA3 readout, thus a new hidden unit with random weights replaces the prior unit. Thereupon another round of perceptron learning is implemented. This simple rule results in a markedly reduced generalization error at the CA3 readout that continues to decline with each subsequent round of neurogenesis. Intuitively, this neurogenesis improves linear separability in the DG space allowing the CA3 readout to draw a decision hyperplane with a greater margin between the classes. This form of neuronal turnover may represent a biologically plausible replacement for error backpropagation.

Pattern classification by memristive crossbar circuits using ex situ and in situ training

Memristors are memory resistors that promise the efficient implementation of synaptic weights in artificial neural networks. Whereas demonstrations of the synaptic operation of memristors already exist, the implementation of even simple networks is more challenging and has yet to be reported. Here we demonstrate pattern classification using a single-layer perceptron network implemented with a memrisitive crossbar circuit and trained using the perceptron learning rule by ex situ and in situ methods. In the first case, synaptic weights, which are realized as conductances of titanium dioxide memristors, are calculated on a precursor software-based network and then imported sequentially into the crossbar circuit. In the second case, training is implemented in situ, so the weights are adjusted in parallel. Both methods work satisfactorily despite significant variations in the switching behaviour of the memristors. These results give hope for the anticipated efficient implementation of artificial neuromorphic networks and pave the way for dense, high-performance information processing systems.

Proposing Effort Estimation of COCOMO II through Perceptron Learning Rule

The software cost estimation is now one of centre of attention for computer software industries. As software industry runs many projects simultaneously they have to prioritize different processes based on time, cost, and number of staff, sequentially. With the increasing complexity of software, the cost of the software development is also increasing. So it is required to rely on the effective techniques for estimating software costs. Accurate cost estimation is needed because it can help to prioritize and classify development projects. In this paper, the most popular software cost estimation model, COCOMO II (post architecture model of COCOMO), is discussed. The estimation of COCOMO II is enhanced through neural network. The network is trained trough perceptron learning rule. The company’s previous projects dataset of estimation and actual cost can be used to train the network. The cost estimation result of COCOMO II is compared with trained network.

A New Supervised Learning Algorithm for Spiking Neurons

The purpose of supervised learning with temporal encoding for spiking neurons is to make the neurons emit a specific spike train encoded by the precise firing times of spikes. If only running time is considered, the supervised learning for a spiking neuron is equivalent to distinguishing the times of desired output spikes and the other time during the running process of the neuron through adjusting synaptic weights, which can be regarded as a classification problem. Based on this idea, this letter proposes a new supervised learning method for spiking neurons with temporal encoding; it first transforms the supervised learning into a classification problem and then solves the problem by using the perceptron learning rule. The experiment results show that the proposed method has higher learning accuracy and efficiency over the existing learning methods, so it is more powerful for solving complex and real-time problems.

Hardware Realization ofArtificialNeural Networks Using Analogue Devices(English)

In this work the analogue neural network has been realized by electronic devices likeOperational Amplifiers and Field Effect Transistor (FET). The FET transistor has been utilized to self adjust weight function for neural network. By use of drain and source resistanceR_ds as function of applied voltage on the gate in linear characteristic region , this resistance has been connected to the input of Operational Amplifiers which becomes as weight function of neural network. Implementing these mentioned characteristics for both FET transistor and operational amplifier , the analogue neural network structure like neuron , weights function and activation function have been realized individually by using the National Instrumentmultisim 10 (NI) Software,then the analogue neural network has been trained successfully by using supervised learning rule like single layer perceptron learning rule and delta learning rule. The results, show good fulfillment of a neural network with analogue hardware devices and verifying the learning rules to train network. Keywords:Analogue neural network, BP learning rule,Perceptron learning rule, NI circuit design suits software(NI multisim 10 software)

Determinants of associative memory performance in spiking and non-spiking neural networks with different synaptic plasticity regimes

The majority of experimental studies of synaptic plasticity in the brain have focussed on modifiable connections between excitatory neurons. However, recent developments in the area of inhibitory synaptic plasticity are challenging the view that inhibitory interneurons provide a purely regulatory role [1]. One such example is the observation that endocannabinoid mediated plasticity in GABAergic synapses may be source and target specific (both in terms of identity and location of pre and postsynaptic neurons), allowing for the suppression of inhibition in specific cortical layers and cell types [2]. Moreover, it has recently been shown that inhibitory synaptic plasticity can help balance excitation and inhibition and support the storage and retrieval of memories in recurrent neural networks [3]. In the present study, we explore the effect of different types of synaptic plasticity and other neuronal characteristics on memory performance in neural networks with excitatory and inhibitory subpopulations. In an initial set of simulations, we investigate the performance of networks of perceptrons with a variety of permitted synaptic plasticity regimes. We then extend this work using similarly configured networks of Izhikevich neurons, and we study how the capacity of these networks is affected by synaptic parameters and neuronal firing patterns. The networks are trained using a modified version of the perceptron learning rule that enforces sign constraints, allowing model neurons to be labelled as excitatory or inhibitory. The retrieval of memories is tested by setting the initial state of the network to a corrupted version of the training pattern. In the non-spiking networks the network is allowed to settle into a steady state. In the networks of spiking neurons, we run the simulation for a fixed period of time and measure the performance by moving a sliding window across the firing trace to produce a set of firing vectors. These are compared to the uncorrupted training pattern and the firing vector with the highest similarity is selected. This process is carried out under increasing load (numbers of stored patterns) until the mean similarity across all tested patterns at a particular load p is < 95%. The effective capacity (the performance metric) is then said to be p – 1. Although a sufficient degree of synaptic plasticity in the networks is required for them to act as associative memories, we find that a lack of plasticity can be tolerated in subsets of synapses. In particular, networks without plasticity at inhibitory-to-inhibitory synapses still perform well, with an effective capacity that is influenced by the weight of the non-plastic connections. Moreover, in the networks of spiking neurons the performance is strongly dependent on synaptic parameters such as decay and scaling, the types of neurons used in the inhibitory and excitatory subpopulations (such as regular or fast spiking neurons), and the sparsity of the training patterns.

Statistical Mechanics of On-line Ensemble Teacher Learning through a Novel Perceptron Learning Rule

In ensemble teacher learning, ensemble teachers have only uncertain information about the true teacher, and this information is given by an ensemble consisting of an infinite number of ensemble teachers whose variety is sufficiently rich. In this learning, a student learns from an ensemble teacher that is iteratively selected randomly from a pool of many ensemble teachers. An interesting point of ensemble teacher learning is the asymptotic behavior of the student to approach the true teacher by learning from ensemble teachers. The student performance is improved by using the Hebbian learning rule in the learning. However, the perceptron learning rule cannot improve the student performance. On the other hand, we proposed a perceptron learning rule with a margin. This learning rule is identical to the perceptron learning rule when the margin is zero and identical to the Hebbian learning rule when the margin is infinity. Thus, this rule connects the perceptron learning rule and the Hebbian learning rule continuously through the size of the margin. Using this rule, we study changes in the learning behavior from the perceptron learning rule to the Hebbian learning rule by considering several margin sizes. From the results, we show that by setting a margin of kappa > 0, the effect of an ensemble appears and becomes significant when a larger margin kappa is used.

C-Mantec: A novel constructive neural network algorithm incorporating competition between neurons

C-Mantec is a novel neural network constructive algorithm that combines competition between neurons with a stable modified perceptron learning rule. The neuron learning is governed by the thermal perceptron rule that ensures stability of the acquired knowledge while the architecture grows and while the neurons compete for new incoming information. Competition makes it possible that even after new units have been added to the network, existing neurons still can learn if the incoming information is similar to their stored knowledge, and this constitutes a major difference with existing constructing algorithms. The new algorithm is tested on two different sets of benchmark problems: a Boolean function set used in logic circuit design and a well studied set of real world problems. Both sets were used to analyze the size of the constructed architectures and the generalization ability obtained and to compare the results with those from other standard and well known classification algorithms. The problem of overfitting is also analyzed, and a new built-in method to avoid its effects is devised and successfully applied within an active learning paradigm that filter noisy examples. The results show that the new algorithm generates very compact neural architectures with state-of-the-art generalization capabilities.