Types Of Learning Algorithms Research Articles

Застосування інтелектуальної діагностичної інфраструктури для управління кібербезпекою

How use of structural features in the construction of hybrid models allows supporting of models adjustment and adaptation to problem-subject environment was considered in the article. The following features were attributed to structural ones: the type of learning algorithm; kind of activation function; the number of layers of the neural network; type of neurons; way of spreading information in neural networks; method of evaluating and interpreting the results of the neural network; the format of fuzzy inference rules; fuzzification and defuzzification method; way to implement the operations of fuzzy implication and logical operations NOT, AND, OR; kind of used genetic operators and the target functions, etc. We propose to use a neural network approach as a basis for the decision of difficulty tasks using decision-support systems. Its effectiveness can be enhanced by: prior training or adjustment of individual neural modules for solvable problem; incorporation of knowledge about the peculiarities of the domain in the hierarchical (multilayer) neural networks structure; application of basic types of hybrid models in which neural network communicates with other information technologies.

Programmable Spike-Timing-Dependent Plasticity Learning Circuits in Neuromorphic VLSI Architectures

Hardware implementations of spiking neural networks offer promising solutions for computational tasks that require compact and low-power computing technologies. As these solutions depend on both the specific network architecture and the type of learning algorithm used, it is important to develop spiking neural network devices that offer the possibility to reconfigure their network topology and to implement different types of learning mechanisms. Here we present a neuromorphic multi-neuron VLSI device with on-chip programmable event-based hybrid analog/digital circuits; the event-based nature of the input/output signals allows the use of address-event representation infrastructures for configuring arbitrary network architectures, while the programmable synaptic efficacy circuits allow the implementation of different types of spike-based learning mechanisms. The main contributions of this article are to demonstrate how the programmable neuromorphic system proposed can be configured to implement specific spike-based synaptic plasticity rules and to depict how it can be utilised in a cognitive task. Specifically, we explore the implementation of different spike-timing plasticity learning rules online in a hybrid system comprising a workstation and when the neuromorphic VLSI device is interfaced to it, and we demonstrate how, after training, the VLSI device can perform as a standalone component (i.e., without requiring a computer), binary classification of correlated patterns.

Statistical analysis of big data: an approach based on support vector machines for classification and regression problems

A new type of learning algorithms with the supervisor for estimating multidimensional functions is considered. These methods based on Support Vector Machines are widely used due to their ability to deal with high-dimensional and large datasets, and their flexibility in modeling diverse sources of data. Support vector machines and related kernel methods are extremely good at solving prediction problems in computational biology. A background about statistical learning theory and kernel feature spaces is given including practical and algorithmic considerations.

Identification of Suitable Learning Algorithm for Neural Network based On-Line Economic Load Dispatch Problem

This paper identifies suitable learning algorithm for neural network based Economic Load Dispatch (ELD). ELD is an important problem in real time power system planning and Operation. The conventional methods used for economic load dispatch are iterative techniques & takes longer time for computation. Neural Network (NN) provides an alternate solution for on-line Load Dispatch. The on-line Load Dispatch requires the NN model to be accurate, simple and structurally compact to ensure faster execution time for effective load dispatch. This in turn to a large extent depends on the type of Neural Learning algorithms used to train the Neural Architecture. The feed forward neural architecture is trained off-line using three types of learning algorithms to solve economic load dispatch problem. Their performance is compared in terms of accuracy, computational complexity and structural compactness. The results are validated for IEEE 26 Bus system. The promising results obtained are presented.

Hermite-neural-network-based adaptive control for a coupled nonlinear chaotic system

A TSK-type Hermite neural network (THNN) is studied in this paper. Since the output weights of the THNN use a functional-type form, it provides powerful representation, high learning performance and good generalization capability. Then, a Hermite-neural-network-based adaptive control (HNNAC) system which is composed of a neural controller and a robust compensator is proposed. The neural controller utilizes a THNN to online approximate an ideal controller, and the robust compensator is designed to eliminate the effect of the approximation error introduced by the neural controller upon the system stability. Moreover, a proportional-integral (PI)-type learning algorithm is derived to speed up the convergence of the tracking error. Finally, the proposed HNNAC system is applied to synchronize a coupled nonlinear chaotic system. In the simulation study, it shows that the proposed HNNAC system can achieve favorable synchronization performance without requiring a preliminary offline tuning.

Convergence of Learning Algorithms in Neural Networks for Adaptive Identification of Nonlinearly Parameterized Systems

This paper deals with studying the asymptotical properties of neural networks used for the adaptive identification of nonlinearly parameterized system. To update the neural network's parameters, simple online gradient type learning algorithm is employed. A distinguishing feature of this algorithm is that its step size remains constant both in non-stochastic case and in stochastic case, and the learning set is infinite. Based on the Lyapunov-like concept, sufficient conditions guaranteeing the convergence of the learning algorithms are derived. Simulations are presented to support the theoretical results.

Hydrometeorological Parameters in Prediction of Soil Temperature by Means of Artificial Neural Network: Case Study in Wyoming

AbstractThe current study was undertaken to analyze the performance of three back-propagation training algorithms of artificial neural network (ANN) along with a multiple linear regression model (MLR) for transient simulation of short/midterm soil temperature (TS). Each ANN used a different type of learning algorithm (gradient descent, conjugate gradient, and Levenberg-Marquardt) as methods for daily and weekly TS prediction. The analysis was performed as a case study using three meteorological parameters [air temperature (TA), net radiation (NR), and relative humidity (RH)] and two hydrological variables [precipitation (P) and runoff (Q)] as input data for a region in Wyoming. Pearson and cross correlation analyses were applied to investigate the relationships between input and target values to determine several input combinations. The correlation analysis indicated that the meteorological parameters of TA and NR were reasonably correlated with TS. Based on the input combinations, 14 models were construc...

Learning Algorithms for Fuzzy Cognitive Maps—A Review Study

This study presents a survey on the most recent learning approaches and algorithms that are related to fuzzy cognitive maps (FCMs). FCMs are cognition fuzzy influence graphs, which are based on fuzzy logic and neural network aspects that inherit their main advantages. They gained momentum due to their dynamic characteristics and learning capabilities. These capabilities make them essential for modeling and decision-making tasks as they improve the performance of these tasks. An efficient number of learning algorithms for FCMs, by modifying the FCM weight matrix, have been developed in order to update the initial knowledge of human experts and/or include any knowledge from historical data in order to produce learned weights. The proposed learning techniques have mainly been concentrated on three directions: on the production of weight matrices on the basis of historical data, on adaptation of the cause-effect relationships of the FCM on the basis of experts' intervention, and on the production of weight matrices by combining experts' knowledge and data. The learning techniques could be categorized into three groups on the basis of the learning paradigm: Hebbian-based, population-based, and hybrid, which subsequently combine the main aspects of Hebbian-based- and population-based-type learning algorithms. These types of learning algorithms are the most efficient and widely used to train the FCMs, according to the existing literature. A survey on recent advances on learning methodologies and algorithms for FCMs that present their dynamic capabilities and application characteristics in diverse scientific fields is established here.

Feature extraction for engine fault diagnosis utilizing the generalized S-transform and non-negative tensor factorization

In this study, a novel feature extraction scheme was proposed for engine fault diagnosis utilizing the generalized S-transform combined with the non-negative tensor factorization (NTF). To represent the information of the non-stationary vibration signals acquired from engine, the generalized S-transform was used to get a time–frequency distribution with enhanced energy concentration. Meanwhile, a newly developed technique called NTF, which can preserve more structure information hiding in original two-dimensional matrices compared to the non-negative matrix factorization (NMF), was adopted to extract more informative features from the time–frequency matrices. Five operating states of engine were tested in an experiment for evaluating the proposed feature extraction scheme. Four different types of learning algorithms were employed to conduct the fault classification task. The NMF technique was also used for feature extraction and compared with the NTF approach. The experimental results have demonstrated that the proposed feature extraction scheme can achieve a satisfactory performance when applied to diagnose the engine faults.

Cross-situational and supervised learning in the emergence of communication

Scenarios for the emergence or bootstrap of a lexicon involve the repeated interaction between at least two agents who must reach a consensus on how to name N objects using H words. Here we consider minimal models of two types of learning algorithms: cross-situational learning, in which the individuals determine the meaning of a word by looking for something in common across all observed uses of that word, and supervised operant conditioning learning, in which there is strong feedback between individuals about the intended meaning of the words. Despite the stark differences between these learning schemes, we show that they yield the same communication accuracy in the limits of large N and H, which coincides with the result of the classical occupancy problem of randomly assigning N objects to H words. Key words: lexicon bootstrapping; cross-situational learning; supervised learning; random occupancy problems

Comparison of Multivariate Regression and Artificial Neural Networks for Peak Urban Water-Demand Forecasting: Evaluation of Different ANN Learning Algorithms

For the past several years, Cyprus has been facing an unprecedented water crisis. Four options that have been considered to help resolve the problem of drought in Cyprus include imposing effective water use restrictions, implementing water-demand reduction programs, optimizing water supply systems, and developing sustainable alternative water source strategies. An important aspect of these initiatives is the accurate forecasting of short-term water demands, and in particular, peak water demands. This study compared multiple linear regression and three types of multilayer perceptron artificial neural networks (each of which used a different type of learning algorithm) as methods for peak weekly water-demand forecast modeling. The analysis was performed on 6 years of peak weekly water-demand data and meteorological variables (maximum weekly temperature and total weekly rainfall) for two different regions (Athalassa and Public Garden) in the city of Nicosia, Cyprus. 20 multiple linear regression models, 20 L...

Different types of learning algorithms of artificial neural network (ANN) models for prediction of gross calorific value (GCV) of coals

Correlations are very significant from earliest days, in some cases, it is essential as it is difficult to measure the amount directly, and in other cases, it is desirable to ascertain the results with other tests through correlations. Soft computing techniques are now being used as alternative statistical tools, and new techniques such as; artificial neural networks, fuzzy inference systems, genetic algorithms, etc. and their hybrid forms have been employed for developing of the predictive models to estimate the needed parameters, in the recent years. Determination of gross calorific value (GCV) of coals is very important to characterize coal and organic shales; it is difficult, expensive, time consuming and is a destructive analysis. In this paper, use of different learning algorithms of artificial neural networks such as MLP, RBF (exact), RBF (k-means) and RBF (SOM) for prediction of GCV was described. As a result of this paper, all models exhibited high performance for predicting GCV. Although the four different algorithms of ANN have almost the same prediction capability, accuracy of MLP has relatively higher than other models. The use of soft computing techniques will provide new approaches and methodologies in prediction of some parameters in the investigations about the fuels. Key words: ANN, MLP, RBF, soft computing, coal, gross calorific value.

Robust wavelet-based adaptive neural controller design with a fuzzy compensator

In this paper, a robust wavelet-based adaptive neural control (RWANC) with a PI type learning algorithm is proposed. The proposed RWANC system is composed of a wavelet neural controller and a fuzzy compensation controller. The wavelet neural control is utilized to approximate an ideal controller and the fuzzy compensation controller with a fuzzy logic system in it is used to remove the chattering phenomena of conventional sliding-mode control completely. In the RWANC, the learning algorithm is derived based on the Lyapunov function, thus the closed-loop system's stability can be guaranteed. The chaotic system control has become an emerging topic in engineering community since the uncontrolled system displays complex, noisy-like and unpredictable behavior. Therefore, the proposed RWANC approach is applied to a second-order chaotic nonlinear system to investigate the effectiveness. Through the simulation results, the proposed RWANC scheme can achieve favorable tracking performance and the convergence of the tracking error and control parameters can be accelerated by the developed PI adaptation learning algorithm.

Simultaneous curve registration and clustering for functional data

Study of dynamic processes in many areas of science has led to the appearance of functional data sets. It is often the case that individual trajectories vary both in the amplitude space and in the time space. We develop a coherent clustering procedure that allows for temporal aligning. Under this framework, closed form solutions of an EM type learning algorithm are derived. The method can be applied to all types of curve data but is particularly useful when phase variation is present. We demonstrate the method by both simulation studies and an application to human growth curves.

HEURISTIC PARTICLE SWARM OPTIMIZATION LEARNING-BASED IMAGE COMPRESSION SYSTEM

Traditional LBG algorithm is a pure iterative optimization procedure to achieve the vector quantization (VQ) codebook, where an initial codebook is continually refined at every iteration to reduce the distortion between code-vectors and a given training data set. However, such interactive type learning algorithms will easily direct final results converging toward the local optimization while the high quality of the initial codebook is not available. In this article, an efficient heuristic-based learning method, called novel particle swarm optimization (NPSO), is proposed to design the proper codebook of VQ scheme that can develop the image compression system. To improve the performance of the basic PSO, the centroid updating machine applies the one step-size gradient descent learning step in the heuristic learning procedure. Additionally, the presented NPSO with advantages of the centroid updating machine is proposed to quickly achieve the near-optimal reconstructive image. For demonstrating the proposed NPSO learning scheme, the image with several horizontal grey bars is first applied to present the efficiency of the NPSO learning mechanism. LBG and NPSO learning methods are also applied to test the reconstructing performance in several type images “Lena,” “Airplane,” “Cameraman”, and “peppers.” In our experiments, the NPSO learning algorithm provides the higher performance than conventional LBG methods in the application of building image compression system.

A Recurrent Fuzzy-Network-Based Inverse Modeling Method for a Temperature System Control

Temperature control by a Takagi-Sugeno-Kang (TSK)-type recurrent fuzzy network (TRFN) designed by modeling plant inverse is proposed in this paper. TRFN is a recurrent fuzzy network developed from a series of TSK-type fuzzy if--then rules, and is characterized by structure and parameter learning. In parameter learning, two types of learning algorithms, the Kalman filter and the gradient descent learning algorithms, are applied to consequent parameters depending on the learning situation. The TRFN has the following advantages when applied to temperature control problems: 1) high learning ability, which considerably reduces the controller training time; 2) no a priori knowledge of the plant order is required, which eases the design process; 3) good and robust control performance; 4) online learning ability, i.e., the TRFN can adapt itself to unpredictable plant changes. The TRFN-based direct inverse control configuration is applied to a real water bath temperature control plant, where various control conditions are experimented. The same experiments are also performed by proportional-integral (PI), fuzzy, and neural network controllers. From comparisons, the aforementioned advantages of a TRFN have been verified

Robust intelligent tracking control with PID-type learning algorithm

This paper proposes a robust intelligent tracking controller (RITC) for a class of unknown nonlinear systems. The proposed RITC system is comprised of a neural controller and a robust controller. The neural controller is designed to approximate an ideal controller using a proportional–integral–derivative (PID)-type learning algorithm in the sense of Lyapunov function, and the robust controller is designed to achieve L 2 tracking performance with desired attenuation level. Finally, to investigate the effectiveness of the RITC system, the proposed design methodology is applied to control two chaotic dynamical systems. The simulation results verify that the proposed RITC system using PID-type learning algorithm can achieve faster convergence of the tracking error and controller parameters than that using I-type learning algorithm.

Support-vector-based fuzzy neural network for pattern classification

Fuzzy neural networks (FNNs) for pattern classification usually use the backpropagation or C-cluster type learning algorithms to learn the parameters of the fuzzy rules and membership functions from the training data. However, such kinds of learning algorithms usually cannot minimize the empirical risk (training error) and expected risk (testing error) simultaneously, and thus cannot reach a good classification performance in the testing phase. To tackle this drawback, a support-vector-based fuzzy neural network (SVFNN) is proposed for pattern classification in this paper. The SVFNN combines the superior classification power of support vector machine (SVM) in high dimensional data spaces and the efficient human-like reasoning of FNN in handling uncertainty information. A learning algorithm consisting of three learning phases is developed to construct the SVFNN and train its parameters. In the first phase, the fuzzy rules and membership functions are automatically determined by the clustering principle. In the second phase, the parameters of FNN are calculated by the SVM with the proposed adaptive fuzzy kernel function. In the third phase, the relevant fuzzy rules are selected by the proposed reducing fuzzy rule method. To investigate the effectiveness of the proposed SVFNN classification, it is applied to the Iris, Vehicle, Dna, Satimage, Ijcnn1 datasets from the UCI Repository, Statlog collection and IJCNN challenge 2001, respectively. Experimental results show that the proposed SVFNN for pattern classification can achieve good classification performance with drastically reduced number of fuzzy kernel functions.

Co-retrieval: a boosted reranking approach for video retrieval

Video retrieval compares multimedia queries to items in a video collection in multiple dimensions and combines all the similarity scores into a final retrieval ranking. Although text is the most reliable feature for video retrieval, features from other modalities can provide complementary information. A reranking framework for video retrieval to augment text feature based retrieval with other evidence is presented. A boosted reranking algorithm called co-retrieval is then introduced, which combines a boosting type learning algorithm and a noisy label prediction scheme to select automatically the most useful (weak) features from multiple modalities. The proposed approach is evaluated with queries and video from the 65 h test collection of the 2003 NIST TRECVID evaluation and it achieves considerable improvement over several baseline retrieval algorithms.

Forecasting stock market movement direction with support vector machine

Support vector machine (SVM) is a very specific type of learning algorithms characterized by the capacity control of the decision function, the use of the kernel functions and the sparsity of the s...