Learning Rate Of Neural Network Research Articles

Artifical Neural Network for Forecasting Electricity Consumption in Energy Enterprises

The concept of "Digital Transformation 2030", which defines the national goals and strategic objectives of the development of the Russian Federation for the period up to 2030, specifies specialized goals and objectives. These goals and objectives are an important message for the introduction of intelligent information management systems based on digital technologies in the electric power industry in general and energy consumption in particular. The main challenges for the transition to digital transformation are: increasing the rate of tariff growth for the end consumer; increasing deterioration of the network infrastructure; the presence of excessive network construction; increasing requirements for the quality of energy consumption. Based on the above, this article is devoted to the development of a method for the formation of an intelligent con­trol system at an energy enterprise by obtaining an early forecast of the amount of electricity required. The predicted values will help not only to increase the energy efficiency of the company through the implementation of specialized energy saving measures, but also to reduce the financial costs of electricity. The solution to this problem is presented in the form of an intelligent neural network system. The main advantages of this artificial neural network are versatility, high-speed and accurate learning, as well as a small amount of training data. The artificial neural network itself is based on the freely distributed TensorFlow machine learning software library, and a modified error backpropagation method is used as training, the main difference of which is the addition of an artificial neural network learning rate increase factor. The results of the analysis of the effectiveness of the method showed that the proposed intelligent neural network system is more accurate (including relative to similar solutions): the average error does not exceed 3.08 % over the entire time of the experiment. This will allow companies to carry out energy saving measures, which will be especially useful in the current economic realities.

Using the Yolov5 Neural Network to Recognize the Presence of Personal Protective Equipment

The article discusses the application of the YOLO neural network for the automatic detection and identification of personal protective equipment for enterprise personnel in digital images. A comparative analysis of various YOLOv5 network architectures is given, depending on the neural network learning rate and recognition accuracy. The tools for training a neural network in Python within the Google Colab environment, which allows efficient use of remote computing power of GPUs, are considered. An example of network training based on collected dataset with six classes for recognizing the presence of personal protective equipment (with a mask; without a mask; the mask is worn incorrectly; with a head protector; without a head protector; special clothing) is considered. The settings of the YOLOv5 neural network and recommendations for improving object recognition by supplementing the dataset with certain condition images are described. Four variants of network training and different number of epochs were produced using the YOLOv5s and YOLOv5l architecture. The optimal neural network architecture and the number of required epochs has been selected. The concept of average accuracy (Mean Average Precision - mAP) is considered and a comparative analysis of the prediction accuracy from the selected network architecture and settings is given. A practical example of recognizing the presence of personal protective equipment in digital images using a trained YOLOv5 neural network is given. On average, the recognition accuracy is 0.7, which is not a bad result for solving this class of problems. The paper deals with issues of improving the accuracy of object recognition. Plans for the detection of previously detected and new classes of personal protective equipment are given, as well as confirmation of the fact that the personnel of enterprises have passed the regulatory procedures based on digital video sequence.

Research of Neural Network Learning Rate Control Methods for the Automatic Signal Type Recognition

A neural network with dynamic parameters and algorithms for the learning rate hyperparameter control has been developed in the LabVIEW software environment in order to increase the efficiency of the signal type recognition algorithm and reduce the network training time. The learning rate hyperparameter control methods testing results are presented. As a result, the most effective method for this task was selected.

Vibration Control of a Constrained Two-Link Flexible Robotic Manipulator With Fixed-Time Convergence.

With the more extensive application of flexible robots, the expectation for flexible manipulators is also increasing rapidly. However, the fast convergence will cause the increase of vibration amplitude to some extent, and it is difficult to obtain vibration suppression and satisfactory transient performance at the same time. In order to deal with the problem, a fixed-time learning control method is proposed to realize the fast convergence. The constraint on system outputs, system uncertainty, and input saturation is addressed under the fixed-time convergence framework. A novel adaptive law for neural networks is integrated into the backstepping method, which enhances the learning rate of neural networks. The imposed constraint on the vibration amplitude is guaranteed by using the barrier Lyapunov function (BLF). Moreover, the chattering problem is addressed by approximating the sign function smoothly. In the end, some simulations have been carried out to show the effectiveness of the proposed method.

Event-Based Projective Synchronization for Different Dimensional Complex Dynamical Networks with Unknown Dynamics by Using Data-Driven Scheme

In this paper, the projective synchronization problem for different dimensional complex networks (CNs) with unknown dynamics is investigated. First, by selecting a projective matrix, the error system is obtained and an event-based projective synchronization control policy is proposed to realize the projective synchronization between two complex networks with different dimensions. It is revealed that the projective synchronization problem can be transformed into the optimal regulation of the error system with a performance function. Then, a data-driven control scheme is proposed to implement the event-trigged projective synchronization control policy, which is composed of identifier, critic network and actuator. The identifier is applied to estimate the unknown dynamics. The actuator is employed to construct the control inputs and the optimal value is estimated by the critic network. Both actuator and critic network are based on neural networks. The neural network weights and controller are updated at event-triggered instant so that the computing and communication resources can be saved. By employing appropriate event-triggered threshold and learning rate of neural network, the synchronization error is proved to be be asymptotically approaching zero and the Zeno behaviors are excluded. Finally, a numerical example is given to verify the effectiveness of the obtained results.

Learning-Based WiFi Traffic Load Estimation in NR-U Systems

The unlicensed spectrum has been utilized to make up the shortage on frequency spectrum in new radio (NR) systems. To fully exploit the advantages brought by the unlicensed bands, one of the key issues is to guarantee the fair coexistence with WiFi systems. To reach this goal, timely and accurate estimation on the WiFi traffic loads is an important prerequisite. In this paper, a machine learning (ML) based method is proposed to detect the number of WiFi users on the unlicensed bands. An unsupervised Neural Network (NN) structure is applied to filter the detected transmission collision probability on the unlicensed spectrum, which enables the NR users to precisely rectify the measurement error and estimate the number of active WiFi users. Moreover, NN is trained online and the related parameters and learning rate of NN are jointly optimized to estimate the number of WiFi users adaptively with high accuracy. Simulation results demonstrate that compared with the conventional Kalman Filter based detection mechanism, the proposed approach has lower complexity and can achieve a more stable and accurate estimation.

Event-driven optimal control for uncertain nonlinear systems with external disturbance via adaptive dynamic programming

Abstract This paper investigates the optimal control of uncertain nonlinear systems with external disturbance via adaptive dynamic programming, where two kinds of controllers are introduced and event-driven strategy is used to update the controllers as well as the network weights. One controller is constrained-input, and another is used to offset the external disturbance. Three kinds of neural networks (identifier network, critic network and actor network) are used to approximate the uncertain dynamics, the optimal value, control inputs and external disturbance, respectively. To the best knowledge of the authors, the advantage of this paper is that almost all the existing systems about optimal control via adaptive dynamic programming can be considered as special cases of this system, where the external disturbance is considered. By designing appropriate parameters, the uncertain nonlinear system is asymptotically stable with the event-driven controller. And, under the given learning rates of neural networks, the system state and the estimation error of neural networks are proved to be uniform ultimate boundedness (UUB) via adaptive dynamic programming. Finally, numerical simulation results are presented to verify the theoretical analysis.

Learning Step Size Controllers for Robust Neural Network Training

This paper investigates algorithms to automatically adapt the learning rate of neural networks (NNs). Starting with stochastic gradient descent, a large variety of learning methods has been proposed for the NN setting. However, these methods are usually sensitive to the initial learning rate which has to be chosen by the experimenter. We investigate several features and show how an adaptive controller can adjust the learning rate without prior knowledge of the learning problem at hand.

About Heating Plants Control System Developing on Basis of Neural Network Usage for PID-Regulator Parameters Optimization

An adaptive control system implementation is described. Such system is based on a neural network, used for online PID-regulator coefficients tuning. The backpropagation online training method is used. It is modified by adding a rule base. It contains conditions on choosing neural network learning rate. PID-regulator with neural tuner and conventional PID-regulator were used as regulators during the process of heating furnace control modeling. Such experiments were made for different loading furnace modes and setpoint schedules. The 11% economy of time on setpoint schedule realizing was achieved with the help of proposed neural tuner..

An Enhanced Approach for Classification in Web Usage Mining using Neural Network Learning Algorithms for Supervised Learning

ABSTRACT Now a day data on the web is growing by a rapid speed, large volume of data is available on web. So, extract useful knowledge from large web data, efficient web mining methods are required to handle those data and achieve various functionalities such a as user trend analysis, web profile analysis, Web AD market change analysis, etc. The concept of neural network helps to handle large volume of data by its characteristics [1]. Several neural network learning algorithms provides better supervised learning. They are capable to handle huge dynamic data. Specially, LVQ (Learning Vector Quantization) algorithms are useful for supervised, dynamic labeling or post-training map labeling and supervised version of SOM through that the approximation of distribution of class with less number of codebook vectors and able to minimize classification errors respectively [9]. MLVQ and HLVQ are both the techniques are following a concept of multi pass in which more than one pass can be performed on the same model and is very useful for gaining best desired results [11]. Here in this paper, we are going to discuss a new technique that will work on hierarchical as well as multi pass approach that is having the advantages of both multi pass and hierarchical approach by combining the benefits of both and designed a new algorithm. That new technique will more accurate, less time consuming and able to decrease learning rate for neural network. The basic HLVQ approach will follow the same algorithm for generation of all phases. In addition to that HMLVQ provides better efficiency through enhancing advantages the various approaches for the same classification process.

Comparison of Early Stopping Criteria for Neural-Network-Based Subpixel Classification

A neural-network-based subpixel classification is one of the most commonly used approaches to address spectral mixture problems. Neural-network subpixel-classification performance is directly related to the network-training protocols used. This letter examined early stopping criteria for network training of subpixel land-cover classification. A new stopping criterion is proposed that is based on the reduction of mean squared error (MSE) for a validation data set. We obtained excellent results by stopping the network training when the reduction of MSE between training iterations became marginal. Furthermore, the neural-network learning rate can be used as a threshold value to identify the stopping point. The approach appeared to be robust for both simulation data and actual remote-sensing data. Use of this criterion outperformed two other commonly used stopping criteria: a predefined number of training iterations and a cross-validation approach.

The Neural-Network Approaches to Solve Nonlinear Equation

In this paper, we proposed two neural - network a pproaches for solving nonlinear equations or polynomials . The first method is suitable for finding simple roots of nonlinear equation or polynomial, and the second approach is fit to finding both the multiple and simple roots of nonlinear equations or polynomial, which were not well solved by the other methods . The convergence of algorithm proposed was researched. The convergence theorem provides the theory criterion selecting learning rate of neural network. The specific examples showed that the proposed method can find the simple or multiple roots of nonlinear equations or polynomials at a very rapid convergence and very high accuracy with less computation.

ANN-Based Adaptive Control of Robotic Manipulators With Friction and Joint Elasticity

This paper proposes a control strategy based on artificial neural networks (ANNs) for a positioning system with a flexible transmission element, taking into account Coulomb friction for both motor and load, and using a variable learning rate for adaptation to parameter changes and accelerate convergence. A control structure consists of a feedforward ANN that approximates the manipulator's inverse dynamical model, an ANN feedback control law, a reference model, and the adaptation process of the ANNs with a variable learning rate. A supervisor that adapts the neural network's learning rate and a rule-based supervisor for online adaptation of the parameters of the reference model are proposed to maintain the stability of the system for large variations of load parameters. Simulation results highlight the performance of the controller to compensate the nonlinear friction terms, particularly Coulomb friction, and flexibility, and its robustness to the load and drive motor inertia parameter changes. Internal stability, which is a potential problem in such a system, is also verified. The controller is suitable for DSP and very large scale integration implementation and can be used to improve static and dynamic performances of electromechanical systems.

A Control Strategy of Levelling Load Power Fluctuation Based on Fuzzy-Neural Network by Tuning Coefficients of Learning Rate with Genetic Algorithm.

The effective usage of the power facilities can be realized by leveling the fluctuating active power and compensating the reactive power. The fuzzy and fuzzy-neural network control strategy of superconducting magnet energy storages (SMES) was proposed for this purpose. The control results depend on the values of coefficients of learning rate in fuzzy-neural network. Therefore, it is desired to obtain better control results that the coefficients of learning rate are tuned to the optimum value.In this paper, the control strategy based on an auto-tuning of scaling factors with neural network and tuning of coefficients of learning rate of neural network with genetic algorithm is proposed for leveling load fluctuation. Encoding and decoding of coefficients of learning rate and selection, crossover and mutation of genetic operation are shown and crossover rate, mutation rate is discussed. Then, we can achieve the better leveling of load power fluctuation by using fuzzy-neural network with genetic algorithm.

Improving the learning rate of neural networks: [formula omitted]. Department of Computer Science, Rutgers University, New-Brunswick, NJ 08903 USA

Improving the learning rate of neural networks: [formula omitted]. Department of Computer Science, Rutgers University, New-Brunswick, NJ 08903 USA