Variable Learning Rate Research Articles

Analysing the influence of trade, technology learning and policy on the employment prospects of wind and solar energy deployment: The EU case

Job creation is arguably an important socioeconomic benefit of renewable energy deployment. In turn, this employment creation may be contingent upon the influence of some key factors, including technology learning, trade effects and policies and may affect different renewable energy technologies and activities of the renewable energy value chain in different ways. This paper estimates the gross employment stemming from the deployment of three renewable electricity technologies – photovoltaics (PV), wind on-shore and wind off-shore – up to 2050 for all Member States of the European Union. It uses a novel analytical methodology which is able to capture the influence of technology learning and internal and external trade. Additionally, it provides highly disaggregated results per activity in the supply chain (manufacturing, installation and O&M), year and country for different technology and policy scenarios. The results show that the employment created by those three technologies can be significant but considerable differences across technologies, activities and countries can be observed. In the analyzed period (2014–2050), most employment will be created in the PV sector, in the operation and maintenance activities and it will be highly geographically concentrated in a few countries. However, job creation will strongly depend on the scenarios and assumptions being made. In particular our findings suggest that the availability of carbon capture and storage will have a considerable influence on the number of jobs being created. In contrast, changes in other assumptions have limited effects on the results: a variable (vs. a constant) learning rate, more restrictive emissions targets by 2050 and higher PV costs.

Coarse-Fine Self-Learning Active Mass Damper for Frequency Tracking Vibration Control

This paper presents a coarse-fine self-learning active mass damper (AMD) for frequency tracking vibration control. Even though the AMD is under unknown disturbance, it can obtain the exact exciting frequency of primary system after several self tries and achieve significant vibration suppression. The frequency tuning algorithm consists of two improved techniques, namely, the variable-gain frequency estimator and variable learning rate [Formula: see text]-learning algorithm, both aim to accelerate the convergence speed of frequency estimation. The coarse tuning method helps to tune the AMD’s natural frequency in a relatively wide effective bandwidth using frequency estimator and the fine tuning method can achieve precise tune in a tiny-scale bandwidth. The simulation results demonstrate that this coarse-fine learning AMD can help obtain the exact exciting frequency after several times of self-learning, and the vibration of primary system is dramatically attenuated by tuning the AMD’s natural frequency to match with the exciting frequency. This is the first time for the tuning method of mass damper proposed using machine learning, which can help obtain the exact exciting frequency after several self-tries.

A Batch Variable Learning Rate Gradient Descent Algorithm With the SmoothingL1/2Regularization for Takagi-Sugeno Models

A batch variable learning rate gradient descent algorithm is proposed to efficiently train a neuro-fuzzy network of zero-order Takagi-Sugeno inference systems. By using the advantages of regularization, the smoothing L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1/2</sub> regularization is utilized to find more appropriate sparse network. Combining the second-order information of the smoothing error function, a variable learning rate is chosen along the steep descent direction, which avoids line search procedure and may reduce the cost of computation. In order to appropriately adjust the Lipschitz constant of the smoothing error function in the learning rate, a new scheme is proposed by introducing a hyper-parameter. Also the article applies the modified secant equation for estimating the Lipschitz constant, which makes the algorithm greatly reduce the oscillating phenomenon and improve the robustness. Under appropriate assumptions, a convergent result of the proposed algorithm is also given. Simulation results for two identification and classification problems show that the proposed algorithm has better numerical performance and promotes the sparsity capability of the network, compared with the common batch gradient descent algorithm and a variable learning rate gradient-based algorithm.

An integrated neural network with nonlinear output structure for interval-valued data

In this paper, an integrated model combining interval deep belief network (IDBN) and neural network with nonlinear weights, called IDBN-NN, is proposed for interval-valued data modeling. Firstly, the IDBN with variable learning rate is designed to initialize parameters of each sub-model. Based on a modified contrastive divergence algorithm the least square method is adopted to identify the coefficients of nonlinear weights in the output layer. Then, to improve the modeling accuracy, the Fuzzy C-Means (FCM) method and the Particle Swarm Optimization (PSO) algorithm are applied to tune the weights of sub-models. Though each sub-model can capture the nonlinear feature of the original system, by intersecting cut sets the synthesizing modeling scheme can further improve the performance of the proposed model. Some numerical examples show that the IDBN-NN with nonlinear output structure can achieve higher accuracy than some interval-valued data modeling methods.

RETRACTED ARTICLE: Integral Backstepping Control of LPMSM Drive System Using Revised Recurrent Fuzzy NN and Mended Particle Swarm Optimization

A linear permanent magnet synchronous motor (LPMSM) drive system is keeping in many nonlinear effects such as the external load force, the flux saturation, the cogging force, the column friction and Stribeck force, and the parameters variations. Due to the uncertainty effects the existing linear controllers can not achieve better control performances for the LPMSM drive system. To raise robustness under occurrence of uncertainty, the integral backstepping control system with hitting function is proposed for controlling the LPMSM drive system in accordance with the Lyapunov function. To improve larger chattering phenomenon under uncertainties effects, the integral backstepping control system with revised recurrent fuzzy neural network (RRFNN) and mended particle swarm optimization (MPSO) is proposed to operate the LPMSM drive system to raise robustness of system. The RRFNN is used to estimate the value of the external lumped force uncertainty. Moreover, the error compensation control with the error compensation mechanism is proposed to compensate the minimum reconstructed error of the error estimation law. Besides, four variable learning rates in the weights of the RRFNN are regulated by virtue of MPSO with segment regulation to speed-up parameter’s convergence. Finally, comparative performances through some tentative upshots are verified that the integral backstepping control system by virtue of RRFNN with MPSO has better control performances than those of the proposed methods for the LPMSM drive system.

Application of artificial neural networks for prediction of sinter quality based on process parameters control

According to the characteristics of non-linearity, strong coupling and a large time delay in the sintering process, the overall analysis for the sintering process has been carried out from the process parameter control point. The sinter performance evaluation indexes and the main influential parameters were determined. The quality prediction model for the sintering process was established using back propagation (BP) neural network algorithm with momentum and variable learning rate. The simulation experimental results show that the model has a higher prediction accuracy and a stronger self-learning ability. The predictive hit rate of random samples is over 81% by adopting BP neural network with the structure of 15-24-4 and network error is 0.65×10−3, thereby verifying the accuracy and effectiveness of the quality prediction model on the basis of process parameters control.

Micrometer Backstepping Control System for Linear Motion Single Axis Robot Machine Drive.

In order to cut down influence on the uncertainty disturbances of a linear motion single axis robot machine, such as the external load force, the cogging force, the column friction force, the Stribeck force, and the parameters variations, the micrometer backstepping control system, using an amended recurrent Gottlieb polynomials neural network and altered ant colony optimization (AACO) with the compensated controller, is put forward for a linear motion single axis robot machine drive system mounted on the linear-optical ruler with 1 um resolution. To achieve high-precision control performance, an adaptive law of the amended recurrent Gottlieb polynomials neural network based on the Lyapunov function is proposed to estimate the lumped uncertainty. Besides this, a novel error-estimated law of the compensated controller is also proposed to compensate for the estimated error between the lumped uncertainty and the amended recurrent Gottlieb polynomials neural network with the adaptive law. Meanwhile, the AACO is used to regulate two variable learning rates in the weights of the amended recurrent Gottlieb polynomials neural network to speed up the convergent speed. The main contributions of this paper are: (1) The digital signal processor (DSP)-based current-regulation pulse width modulation (PWM) control scheme being successfully applied to control the linear motion single axis robot machine drive system; (2) the micrometer backstepping control system using an amended recurrent Gottlieb polynomials neural network with the compensated controller being successfully derived according to the Lyapunov function to diminish the lumped uncertainty effect; (3) achieving high-precision control performance, where an adaptive law of the amended recurrent Gottlieb polynomials neural network based on the Lyapunov function is successfully applied to estimate the lumped uncertainty; (4) a novel error-estimated law of the compensated controller being successfully used to compensate for the estimated error; and (5) the AACO being successfully used to regulate two variable learning rates in the weights of the amended recurrent Gottlieb polynomials neural network to speed up the convergent speed. Finally, the effectiveness of the proposed control scheme is also verified by the experimental results.

Switched reluctance motor circuit drive system using adaptive nonlinear backstepping control with mended recurrent Romanovski polynomials neural network and mended particle swarm optimization

AbstractA switched reluctance motor (SRM) circuit drive system caused many nonlinear effects due to convex construction. The linear control methods were hard to achieve good performance for the SRM circuit drive. The adaptive nonlinear backstepping control system using switching function is proposed for controlling the SRM drive system to obtain good performance. To reduce chattering of control effort, the adaptive nonlinear backstepping control system using adaptive law is proposed to estimate the required lumped uncertainty. When the inertia of the counterweight is varying, this proposed method cannot get a satisfactory performance. The adaptive nonlinear backstepping control system using mended recurrent Romanovski polynomials neural network with adaptive law and error‐estimated law is proposed for controlling the SRM drive system to raise robustness of the SRM drive system. Furthermore, two variable learning rates in the mended recurrent Romanovski polynomials neural network are adopted by using mended particle swarm optimization (PSO) algorithm to speed up parameter's convergence.Finally, comparative performances through some experimental results are verified that the proposed control system has better control performances than those of the proposed methods for the SRM drive system.

Retracted: Adaptive nonlinear backstepping control using mended recurrent Romanovski polynomials neural network and mended particle swarm optimization for switched reluctance motor drive system

A switched reluctance motor (SRM) drive system has highly nonlinear uncertainties owing to a convex construction. It is hard for the linear control methods to achieve good performance for the SRM drive system. An adaptive nonlinear backstepping control system using the mended recurrent Romanovski polynomials neural network and mended PSO with an adaptive law and an error estimated law is proposed to estimate the lumped uncertainty and to compensate the estimated error in order to enhance the robustness of the SRM drive system. Additionally, in accordance with the Lyapunov stability theorem, the adaptive law in the mended recurrent Romanovski polynomials neural network and the error estimated law are established. Furthermore, to help improve convergence and to obtain better learning performance, the mended particle swarm optimization (PSO) algorithm is utilized for adjusting the two varied learning rates of the two parameters in the mended recurrent Romanovski polynomials neural network. Finally, some experimental results and a comparative analysis are verified that the proposed control scheme has better control performances for controlling the SRM drive system.

Variable learning rate EASI-based adaptive blind source separation in situation of nonstationary source and linear time-varying systems

In the case of multiple nonstationary independent source signals and linear instantaneous time-varying mixing systems, it is difficult to adaptively separate the multiple source signals. Therefore, the adaptive blind source separation (BSS) problem is firstly formally expressed and compared with tradition BSS problem. Then, we propose an adaptive blind identification and separation method based on the variable learning rate equivariant adaptive source separation via independence (EASI) algorithm. Furthermore, we analyze the scope and conditions of variable-learning rate EASI algorithm. The adaptive BSS simulation results also show that the variable learning rate EASI algorithm provides better separation effect than the fixed learning rate EASI and recursive least-squares algorithms.

A recurrent TSK interval type-2 fuzzy neural networks control with online structure and parameter learning for mobile robot trajectory tracking

This paper focuses on the design of a recurrent Takagi-Sugeno-Kang interval type-2 fuzzy neural network RTSKIT2FNN for mobile robot trajectory tracking problem. The RTSKIT2FNN is incorporating the recurrent frame of internal-feedback loops into interval type-2 fuzzy neural network which uses simple interval type-2 fuzzy sets in the antecedent part and the Takagi-Sugeno-Kang (TSK) type in the consequent part of the fuzzy rule. The antecedent part forms a local internal feedback loop by feeding the membership function of each node in the fuzzification layer to itself. Initially, the rule base in the RTSKIT2FNN is empty, after that, all rules are generated by online structure learning, and all the parameters of the RTSKIT2FNN are updated online using gradient descent algorithm with varied learning rates VLR. Through experimental results, we demonstrate the effectiveness of the proposed RTSKIT2FNN for mobile robot control.

Retracted: SCRIM drive system using adaptive backstepping control and mended recurrent Romanovski polynomials neural network with reformed particle swarm optimization

SummaryDue to air‐gap field harmonic, cogging torque, stator's current time harmonic, and the influence of flux saturation, a six‐phase copper rotor induction motor (SCRIM) drive system has highly nonlinear uncertainties. Thus, the linear control method for the SCRIM drive system is difficult to achieved good performance under the nonlinear uncertainty action. To obtain better control performance, the adaptive backstepping control system using switching function is firstly proposed for controlling the SCRIM drive system to overcome the uncertainty influence. With the proposed control system, the SCRIM drive system holds in robustness to these uncertainties for the tracking of periodic reference trajectories. To enhance the robustness of the SCRIM drive system, the adaptive backstepping control system using adaptive law is proposed for estimating the required lumped uncertainty to reduce chattering phenomenon. When the inertia of the counterweight is varying, this proposed method can perform well in general situations but cannot get a satisfactory performance. The adaptive backstepping control system using mended recurrent Romanovski polynomials neural network with reformed particle swarm optimization (PSO) is thus proposed to estimate the lumped uncertainty and to compensate estimated error for obtaining better control performance. Furthermore, two variable learning rates of the weights in the mended recurrent Romanovski polynomials neural network are adopted by using reformed PSO to speed up parameter's convergence. Finally, some experimental results with comparative control performances are demonstrated, and then, the effectiveness of proposed control system with better control performance is verified for the position tracking of periodic reference inputs.

Discriminant Zero-Shot Learning with Center Loss

Current work on zero-shot learning (ZSL) generally does not focus on the discriminative ability of the models, which is important for differentiating between classes since our brain focuses on the discriminating part of the object to classify it. For generalized ZSL (GZSL), the fact that the outputs of the model are not comparable leads to a degraded performance. We propose a new ZSL method with a center loss to make the instances from the same class more compact by extracting their discriminative parts. Further, we introduce a varying learning rate to accelerate the model selection process. We also demonstrate how to boost the performance of GZSL by rectifying the outputs of the model to make the outputs be comparable. Experimental results on four benchmarks, including SUN, CUB, AWA2, and aPY, demonstrate the superiority of the proposed method, therein achieving state-of-the-art performance.

Real-Time Event-Triggered Object Tracking in the Presence of Model Drift and Occlusion

In this paper, we propose a novel event-triggered tracking framework for fast and robust visual tracking in the presence of model drift and occlusion. The resulting tracker not only operates in real time, but also is resilient to tracking failures caused by factors such as fast motion and heavy occlusion. Specifically, the tracker consists of an event-triggered decision model as the core module that coordinates other functional modules, including a short-term tracker, occlusion and drift identification, target redetection, short-term tracker updating, and online discriminative learning for a detector. Each functional module is associated with a defined event that is triggered when the proposed conditions are met. The occlusion and drift identification module is intended to perform online evaluation of the short-term tracking. When a model drift event occurs, the target redetection module is activated by the event-triggered decision model to relocate the target and reinitialize the short-term tracker. The short-term tracker updating is carried out at each frame with a variable learning rate depending on the degree of occlusion. A sampling pool is constructed to store discriminative samples that are used to update the detector model. Extensive experiments on large benchmark datasets demonstrate that the proposed tracking algorithm can effectively detect model drift and restore tracking, and more importantly, it outperforms the state-of-the-art approaches in terms of accuracy, efficiency, and robustness.

A SCRIM Drive System Using Backstepping Control and Revamped Recurrent Romanovski PNN with Mended ACO

ABSTRACTA six-phase copper rotor induction motor (SCRIM) drive system with nonlinear uncertainties is controlled by a backstepping control system using a revamped recurrent Romanovski polynomials neural network (PNN). To achieve better control performance, an adaptive law of the revamped recurrent Romanovski PNN is based on the Lyapunov function to estimate the lumped uncertainty. Additionally, an error estimated law is proposed to compensate the estimated error. Meanwhile, the mended ant colony optimization (ACO) is used for regulating two variable learning rates in the revamped recurrent Romanovski PNN. Finally, some experimental results and a comparative analysis are presented to verify the effective of the proposed control system.

Robust adaptive learning approach to self-organizing maps

The Self-Organizing Map (SOM) has attracted much research in diverse fields of study, where it has been successfully applied to a wide range of artificial intelligence applications. In this paper, a new intelligent adaptive learning SOM, in contrast with the conventional SOM algorithm, is presented. The proposed SOM overcomes the disadvantages of the conventional SOM by deriving a new variable learning rate that can adaptively achieve the optimal weights and obtain the winner neurons with a shorter learning process time. To quantify the performance, the proposed SOM was compared with various unsupervised algorithms to examine how well the network weight update equation adapts and reaches the optimum weights with fewer iterations. Extensive experiments were conducted using eight different databases from the UCI and KEEL repositories. The proposed SOM algorithm was further compared with the conventional SOM, GF-SOM, PLSOM, and PLSOM2 algorithms. The proposed SOM algorithm showed superiority in terms of the convergence rate, Quantization Error (QE), Topology Error (TE) preserving map, and accuracy during the classification process.

Online MRAC method using neural networks based on variable learning rate for non-linear systems

This paper presents a new algorithm of a neural network model reference adaptive controller that uses a variable learning rate. It illustrates how the learning rate affects directly training speed. The neural network training algorithms, such as backpropagation, suffer from low-convergence speed as time-consuming. While increasing the learning rate may help to proceed much faster, it can result in unstable training in terms of weights divergence. Therefore, we propose a neural controller training algorithm using a variable learning rate which is capable of speeding up the learning process significantly and it can provide simultaneously stability of the learning process. The results of simulation show that using variable learning rate has better effects both on response time and on tracking performance.

Online MRAC method using neural networks based on variable learning rate for non-linear systems

This paper presents a new algorithm of a neural network model reference adaptive controller that uses a variable learning rate. It illustrates how the learning rate affects directly training speed. The neural network training algorithms, such as backpropagation, suffer from low-convergence speed as time-consuming. While increasing the learning rate may help to proceed much faster, it can result in unstable training in terms of weights divergence. Therefore, we propose a neural controller training algorithm using a variable learning rate which is capable of speeding up the learning process significantly and it can provide simultaneously stability of the learning process. The results of simulation show that using variable learning rate has better effects both on response time and on tracking performance.

Memristor-Based Neural Network Circuit of Full-Function Pavlov Associative Memory With Time Delay and Variable Learning Rate.

Most memristor-based Pavlov associative memory neural networks strictly require that only simultaneous food and ring appear to generate associative memory. In this article, the time delay is considered, in order to form associative memory when the food stimulus lags behind the ring stimulus for a certain period of time. In addition, the rate of learning can be changed with the length of time between the ring stimulus and food stimulus. A memristive neural network circuit that can realize Pavlov associative memory with time delay is designed and verified by the simulation results. The designed circuit consists of a synapse module, a voltage control module, and a time-delay module. The functions, such as learning, forgetting, fast learning, slow forgetting, and time-delay learning, are implemented by the circuit. The Pavlov associative memory neural network with time-delay learning provides a reference for further development of the brain-like systems.

Adaptive Neural Network Control and Optimal Path Planning of UAV Surveillance System With Energy Consumption Prediction

A surveillance system is one of the most interesting research topics for an unmanned aerial vehicle (UAV). However, the problem of planning an energy-efficient path for the surveillance purpose while anticipating disturbances and predicting energy consumptions during the path tracking is still a challenging problem in recent years. The optimal path planning and the disturbance rejection control for a UAV surveillance system are investigated in this paper. A trained and tested energy consumption regression model is used to be the cost function of an optimal path planning scheme, which is designed from a clustered 3D real pilot flight pattern with the proposed K-agglomerative clustering method, and is processed via A-star and set-based particle-swarm-optimization (S-PSO) algorithm with adaptive weights. Moreover, an online adaptive neural network (ANN) controller with varied learning rates is designed to ensure the control stability while having a reliably fast disturbance rejection response. The effectiveness of the proposed framework is verified by numerical simulations and experimental results. By applying the proposed optimal path planning scheme, the energy consumption of the optimal path is only 72.3397 Wh while the average consumed energy of real pilot flight data is 96.593Wh. In addition, the proposed ANN control improves average root-mean-square error (RMSE) of horizontal and vertical tracking performance by 49.083% and 37.50% in comparison with a proportional-integral-differential (PID) control and a fuzzy control under the occurrence of external disturbances. According to all of the results, the combination of the proposed optimal path planning scheme and ANN controller can achieve an energy-efficient UAV surveillance systems with fast disturbance rejection response.