Piecewise-linear modeling of analog circuits using trained feed-forward neural networks and adaptive clustering of hidden neurons

Training of feed-forward neural-networks (FNN) is a challenging nonlinear task in supervised learning systems. Further, derivative learning-based methods are frequently inadequate for the training phase and cause a high computational complexity due to the numerous weight values that need to be tuned. In this study, training of neural-networks is considered as an optimization process and the best values of weights and biases in the structure of FNN are determined by Vortex Search (VS) algorithm. The VS algorithm is a novel metaheuristic optimization method recently developed, inspired by the vortex shape of stirred liquids. VS fulfills the training task to set the optimal weights and biases stated in a matrix. In this context, the proposed VS-based learning method for FNNs (VS-FNN) is conducted to analyze the effectiveness of the VS algorithm in FNN training for the first time in the literature. The proposed method is applied to six datasets whose names are 3-bit XOR, Iris Classification, Wine-Recognition, Wisconsin-Breast-Cancer, Pima-Indians-Diabetes, and Thyroid-Disease. The performance of the proposed algorithm is analyzed by comparing with other training methods based on Artificial Bee Colony Optimization (ABC), Particle Swarm Optimization (PSO), Simulated Annealing (SA), Genetic Algorithm (GA) and Stochastic Gradient Descent (SGD) algorithms. The experimental results show that VS-FNN is generally leading and competitive. It is also said that VS-FNN can be used as a capable tool for neural networks.

The purpose of the research, the results of which are presented in this article, is to determine the possibility and evaluate the effectiveness of using a trained neural network in the diagnosis of ringworm. The article provides an analysis of the methods used for diagnosing dermatomycosis in veterinary practice. One of the actively developing areas at present is the use of artificial neural networks in the diagnosis of animal diseases. The authors have developed a method for diagnosing dermatophytosis using a trained neural network. To identify hair damaged by dermatophyte spores in cats, a trained artificial neural network YOLO v5 was used, based on the YOLO architecture (high-precision artificial neural network), which provides high accuracy and speed of object detection in images. Diagnostics was carried out in three stages. The first stage: the diagnosis of hair in cats damaged by dermatophyte spores was carried out using a trained artificial neural network. The second stage: microscopy by a veterinary specialist of the veterinary center. The third stage: comparison of the received data from the trained artificial neural network and veterinary specialists. Three comparative experiments were carried out on 20 depersonalized samples with different ratios from healthy and sick animals. As a result of testing the trichoscopy method using artificial neural networks for diagnosing spore-damaged hair dermatitis in cats, it was found that a trained artificial neural network of 60 studied samples diagnosed dermatophyte spore damage in 20 samples, a veterinarian - in 17. All positive results were confirmed by a mycological laboratory study. and identification of the pathogen. It has been established that the use of a trained artificial neural network increases the diagnostic efficiency by 15% and reduces the time to perform diagnostic microscopy by 60.3%. The application of the proposed method allows to reduce the time of microscopic examination, improve the accuracy of interpretation of the results, automate methods for identifying causative agents of ringworm in small animals and take timely measures to treat the animal.

Logarithm-approximate floating-point multiplier is applicable to power-efficient neural network training

This paper presents a new technique for automatically creating analog circuit models. The method extracts piecewise linear models from trained neural networks. A model is a set of linear dependencies between circuit performance and design parameters. The paper illustrates the technique for an OTA circuit for which models for gain and bandwidth are generated. As experiments show, the obtained models have simple form that accurately fits the sampled points. These models are useful for fast simulation of systems with nonlinear behavior and performance.

A generalized criterion for the training of feedforward neural networks is proposed. Depending on the optimization strategy used, this criterion leads to a variety of fast learning algorithms for single-layered as well as multilayered neural networks. The simplest algorithm devised on the basis of this generalized criterion is the fast delta rule algorithm, proposed for the training of single-layered neural networks. The application of a similar optimization strategy to multilayered neural networks in conjunction with the proposed generalized criterion provides the fast backpropagation algorithm. Another set of fast algorithms with better convergence properties is derived on the basis of the same strategy that provided recently a family of Efficient LEarning Algorithms for Neural NEtworks (ELEANNE). Several experiments verify that the fast algorithms developed perform the training of neural networks faster than the corresponding learning algorithms existing in the literature.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

A generalized criterion for the training of feedforward neural networks is proposed. Depending on the optimization strategy used, this criterion leads to a variety of fast learning algorithms for single-layered as well as multilayered neural networks. The simplest algorithm devised on the basis of this generalized criterion is the fast delta rule algorithm, proposed for the training of single-layered neural networks. The application of a similar optimization strategy to multilayered neural networks in conjunction with the proposed generalized criterion provides the fast backpropagation algorithm. Another set of fast algorithms with better convergence properties is derived on the basis of the same strategy that provided recently a family of Efficient LEarning Algorithms for Neural NEtworks (ELEANNE). Several experiments verify that the fast algorithms developed perform the training of neural networks faster than the corresponding learning algorithms existing in the literature. >

The accurate of i identificationntrinsically disordered proteins or protein regions is of great importance, as they are involved in critical biological process and related to various human diseases. In this paper, we develop a deep neural network that is based on the well-known VGG16. Our deep neural network is then trained through using 1450 proteins from the dataset DIS1616 and the trained neural network is tested on the remaining 166 proteins. Our trained neural network is also tested on the blind test set R80 and MXD494 to further demonstrate the performance of our model. The MCC value of our trained deep neural network is 0.5132 on the test set DIS166, 0.5270 on the blind test set R80 and 0.4577 on the blind test set MXD494. All of these MCC values of our trained deep neural network exceed the corresponding values of existing prediction methods.

Artificial neural networks (ANN) have been widely used in the field of data classification. Normally, training of neural network is applied with the traditional back propagation technique. As, this approach has various drawbacks, training of neural network is done with Particle Swarm Optimization (PSO). PSO has been widely used to solve the diverse kind of optimization problems. Population initialization performs a significant role in meta-heuristic algorithms. This paper describes a new initialization population approach Log Logistic termed as PSOLL-NN to create the initialization of the swarm. The proposed algorithm has been tested for weight optimization of feed forward neural network; and compared with back propagation Algorithm (BPA), standard PSO (PSO-NN), PSO initialized with Halton Sequence (PSOH-NN), Torus sequence (PSOT-NN) and Sobol sequence (PSOS-NN). The experimental results show that the proposed technique performed exceptionally better than the other traditional techniques. Moreover, the outcome of our work presents a foresight that how the proposed initialization technique can be used as an efficient alternative to standard training approaches for the data classification problems.

Previous studies have shown that factorization and random regrouping significantly improve the performance of the cooperative particle swarm optimization (CPSO) algorithm. However, few studies have examined whether this trend continues when CPSO is applied to the training of feed forward neural networks. Neural network training problems often have very high dimensionality and introduce the issue of saturation, which has been shown to significantly affect the behavior of particles in the swarm; thus it should not be assumed that these trends hold. This study identifies the benefits of random regrouping and factorization to CPSO based neural network training, and proposes a number of approaches to problem decomposition for use in neural network training. Experiments are performed on 11 problems with sizes ranging from 35 up to 32,811 weights and biases, using a number of general approaches to problem decomposition, and state of the art algorithms taken from the literature. This study found that the impact of factorization and random regrouping on solution quality and swarm behavior depends heavily on the general approach to problem decomposition. It is shown that a random problem decomposition is effective in feed forward neural network training. A random problem decomposition has the benefit of reducing the issue of problem decomposition to the tuning of a single parameter.

Training of feed-forward neural networks is a well-known and important hard optimization problem, frequently used for classification purpose. Swarm intelligence metaheuristics have been successfully used for such optimization problems. In this chapter we present how cuckoo search and bat algorithm, as well as the modified version of the bat algorithm, were adjusted and applied to the training of feed-forward neural networks. We used these three algorithms to search for the optimal synaptic weights of the neural network in order to minimize the function errors. The testing was done on four well-known benchmark classification problems. Since the number of neurons in hidden layers may strongly influence the performance of artificial neural networks, we considered several neural networks architectures for different number of neurons in the hidden layers. Results show that the performance of the cuckoo search and bat algorithms is comparable to other state-of-the-art nondeterministic optimization algorithms, with some advantage of the cuckoo search. However, modified bat algorithm outperformed all other algorithms which shows great potential of this recent swarm intelligence algorithm.

Injecting input noise during feedforward neural network (NN) training can improve generalization performance markedly. Reported works justify this fact arguing that noise injection is equivalent to a smoothing regularization with the input noise variance playing the role of the regularization parameter. The success of this approach depends on the appropriate choice of the input noise variance. However, it is often not known a priori if the degree of smoothness imposed on the FNN mapping is consistent with the unknown function to be approximated. In order to have a better control over this smoothing effect, a loss function putting in balance the smoothed fitting induced by the noise injection and the precision of approximation, is proposed. The second term, which aims at penalizing the undesirable effect of input noise injection or controlling the deviation of the random perturbed loss, was obtained by expressing a certain distance between the original loss function and its random perturbed version. In fact, this term can be derived in general for parametrical models that satisfy the Lipschitz property. An example is included to illustrate the effectiveness of learning with this proposed loss function when noise injection is used.

On Fault Tolerant Training of Feedforward Neural Networks

This paper describes a new technique for robust training of feedforward neural networks. The proposed algorithm is employed for the robust neural network training purpose. The quasi-Newton method was studied as one of the most efficient optimization algorithms based on the gradient descent and used as the batch training method of neural networks. On the other hand, the stochastic (online) quasi-Newton method was developed as an algorithm for the machine learning. In this paper the stochastic quasi-Newton training algorithm is improved for robust neural network training. Neural network training for some benchmark problems is presented to demonstrate the proposed algorithm. Furthermore, neural network training for microwave circuit modeling, such as the waveguide and the microstrip examples is presented, demonstrating that the proposed algorithm achieves more accurate models than both the batch and the stochastic quasi-Newton methods.

Hybrid learning schemes for fast training of feed-forward neural networks

This paper presents a new technique for automatically creating analog circuit models. The method extracts - from trained neural networks - piecewise linear models expressing the linear dependencies between circuit performances and design parameters. The paper illustrates the technique for an OTA circuit for which models for gain and bandwidth were automatically generated. The extracted models have a simple form that accurately fits the sampled points and the behavior of the trained neural networks. These models are useful for fast simulation of systems with non-linear behavior and performances.