Synthesis Of Artificial Neural Networks Research Articles

RESEARCH ON THE PRINCIPLES OF ARTIFICIAL NEURAL NETWORK CONSTRUCTION AND PARAMETER MODELING

Studies show that neural networks with multilayer direct propagation have be-come more widespread, which is due to the simplicity of their implementation, the availability of advanced learning methods, and the parallel execution of calculations. In these networks, the computational complexity increases proportionally to the square of the data size, which leads to a decrease in performance and an increase in hardware costs. The main problems of synthesis of artificial neural networks, criteria for evaluating the effectiveness of artificial neural networks, issues of choosing layers and the number of neurons in each layer are investigated. In the work, the mean square error, mean absolute error and mean absolute percentage error criteria are considered for evaluating the efficiency of neural networks. In general, one hidden layer of a neu-ral network is sufficient for most problems, two hidden layers can represent a function of any shape, and adding a third and more layers results in very little improvement in network performance. Expressions are given for the total number of parameters de-pending on the number, and the number of inputs from the binary sequence, as well as the minimum and maximum values of the training examples depending on the total number of parameters and the number of outputs. The studies have shown that the se-lection of the statistical model efficiency evaluation criteria depends on the problem to be solved. The results of the simulations performed using the MATLAB package are presented.

EVOLUTIONARY METHOD FOR SYNTHESIS SPIKING NEURAL NETWORKS USING THE NEUROPATTHERN MECHANISM

Context. The problem of synthesizing pulsed neural networks based on an evolutionary approach to the synthesis of artificial neural networks using a neuropathic mechanism for constructing diagnostic models with a high level of accuracy is considered. The object of research is the process of synthesis of pulsed neural networks using an evolutionary approach and a neuropathic mechanism.
 Objective of the work is to develop a method for synthesizing pulsed neural networks based on an evolutionary approach using a neuropathic mechanism to build diagnostic models with a high level of accuracy of work.
 Method. A method for synthesizing pulsed neural networks based on an evolutionary approach is proposed. At the beginning, a population of pulsed neural networks is generated, and a neuropathic mechanism is used for their encoding and further development, which consists in separate encoding of neurons with different activation functions that are determined beforehand. So each pattern with multiple entry points can define the relationship between a pair of points. In the future, this simplifies the evolutionary development of networks. To decipher a pulsed neural network from a pattern, the coordinates for a pair of neurons are passed to the network that creates the pattern. The network output determines the weight and delay of the connection between two neurons in a pulsed neural network. After that, you can evaluate each neuromodel after evolutionary changes and check the criteria for stopping synthesis. This method allows you to reduce the resource intensity during network synthesis by abstracting the evolutionary changes of the network pattern from itself.
 Results. The developed method is implemented and investigated on the example of the synthesis of a pulsed neural network for use as a model for technical diagnostics. Using the developed method to increase the accuracy of the neuromodel with a test sample by 20%, depending on the computing resources used.
 Conclusions. The conducted experiments confirmed the operability of the proposed mathematical software and allow us to recommend it for use in practice in the synthesis of pulsed neural networks as the basis of diagnostic models for further automation of tasks of diagnostics, forecasting, evaluation and pattern recognition using big data. Prospects for further research may lie in the use of a neuropathic mechanism for indirect encoding of pulsed neural networks, which will provide even more compact data storage and speed up the synthesis process.

Usage of swarm intelligence strategies during projection of parallel neuroevolution methods for neuromodel synthesis

The paper proposes the ways to apply swarm intelligence strategies to parallelize neuroevolution methods for synthesizing artificial neural networks. The proposed approaches will solve a number of problems that usually arise during designing high-performance computing related to the synthesis of neural networks. The object of research is the process of developing a parallel approach for the neuroevolution synthesis of artificial neural networks, namely, the use of swarm intelligence strategies to solve a number of problems in designing a method that would use the resources of a parallel computer system.One of the most problematic areas is the highly adaptive nature and significant operating time of neuroevolution methods. One way to solve these problems is to use parallel computer systems and distributed computing. However, a number of questions arise when designing a parallel neuroevolution method.During research a number of tasks were solved, which included the analysis and study of neuroevolution methods for synthesizing artificial neural networks and problems of their parallelization. Attention is also paid to swarm intelligence methods, which have gained popularity recently and show good results.The new method developed during the work was based on strategies for organizing work with swarm particles. Thus, sub-populations distributed between threads and individuals were analyzed as individual particles that interact with each other and depend on the local environment. Classical genetic operators were modified by criterion mechanisms to improve adaptability.During the experiments, the developed method was compared with classical methods. During the work, special attention was paid not only to the characteristics of the resulting neuromodels, but also to the load on the processor during Operation. The developed method showed acceptable results for all comparisons. The new approach has significantly improved the quality level of the parallel neuroevolution synthesis method, allowing to evenly use the capabilities of computing nodes in a parallel system

MODIFICATION AND PARALLELIZATION OF GENETIC ALGORITHM FOR SYNTHESIS OF ARTIFICIAL NEURAL NETWORKS

Context. The problem of automation synthesis of artificial neural networks for further use in diagnosing, forecasting and pattern recognition is solved. The object of the study was the process of synthesis of ANN using a modified genetic algorithm. Objective. The goals of the work are the reducing the synthesis time and improve the accuracy of the resulting neural network. Method. The method of synthesis of artificial neural networks on the basis of the modified genetic algorithm which can be implementing sequentially and parallel using MIMD – and SIMD-systems is proposed. The use of a high probability of mutation can increase diversity within the population and prevent premature convergence of the method. The choice of a new best specimen, as opposed to a complete restart of the algorithm, significantly saves system resources and ensures the exit from the area of local extrema. The use of new criteria for adaptive selection of mutations, firstly, does not limit the number of hidden neurons, and, secondly, prevents the immeasurable increase in the network. The use of uniform crossover significantly increases the efficiency, as well as allows emulating other crossover operators without problems. Moreover, the use of uniform crossover increases the flexibility of the genetic algorithm. The parallel approach significantly reduces the number of iterations and significantly speedup the synthesis of artificial neural networks. Results. The software which implements the proposed method of synthesis of artificial neural networks and allows to perform the synthesis of networks in sequentially and in parallel on the cores of the CPU or GPU. Conclusions. The experiments have confirmed the efficiency of the proposed method of synthesis of artificial neural networks and allow us to recommend it for use in practice in the processing of data sets for further diagnosis, prediction or pattern recognition. Prospects for further research may consist in the introduction of the possibility of using genetic information of several parents to form a new individual and modification of synthesis methods for recurrent network architectures for big data processing.

Multi-Objective Optimization of Complex Objects in Neural Network Models

Background/Objectives: The study considers the modeling technique that applies artificial neural networks analyzing their types and functional principles. Methods: A comparative analysis of the existing methods of structural and parametric synthesis of artificial neural networks has been carried out; the practicability of applying evolutionary approach to solve this problem has been justified. Findings: The multi-objective optimization of the structure of a neural network model has been formalized, given its computational complexity. The genetic algorithm has been adjusted to solve the problems of unconditional optimization of the parameters of the neural network and of selecting its effective structure in multi-objective setting. The results of solving the practical problem prove that the application of the suggested approach can help alleviate the computational complexity of the obtained structures of artificial neural networks. Applications/Improvements: The results of the study make it possible for a decision maker to select neural network model among multiple options, given the required precision and the available computational resources.

Genetic Representation and Evolvability of Modular Neural Controllers

The manual design of control systems for robotic devices can be challenging. Methods for the automatic synthesis of control systems, such as the evolution of artificial neural networks, are thus widely used in the robotics community. However, in many robotic tasks where multiple interdependent control problems have to be solved simultaneously, the performance of conventional neuroevolution techniques declines. In this paper, we identify interference between the adaptation of different parts of the control system as one of the key challenges in the evolutionary synthesis of artificial neural networks. As modular network architectures have been shown to reduce the effects of such interference, we propose a novel, implicit modular genetic representation that allows the evolutionary algorithm to automatically shape modular network topologies. Our experiments with plastic neural networks in a simple maze learning task indicate that adding a modular genetic representation to a state-of-the-art implicit neuroevolution method leads to better algorithm performance and increases the robustness of evolved solutions against detrimental mutations.

Mixed analog/digital hardware synthesis of artificial neural networks

A methodology for automated synthesis of mixed analog/digital hardware architectures for Artificial Neural Network (ANN) applications is presented, The ANN system is input in the form of a Data Flow Graph (DFG) of operators. The output is a high-level interconnection description of mixed analog/digital building block circuits that is generated by searching a multidimensional design space to satisfy the throughput requirements and minimize an area-time cost function. A combination of quantitative analysis and behavioral modeling techniques tackles the problems of hardware nonidealities by screening suitable analog circuits. Digital hardware is employed when no analog circuit is found that satisfies the performance requirements of any operation. A heuristic partitioning scheme reduces the interfaces between blocks of circuits operating in analog and digital modes. The parallelism in ANN systems is, handled by a new scheduling and allocating procedure that selectively sequentializes groups of operations with the goal of iteratively improving an area-time cost function. The synthesizer is also capable of embedding hardware units that are intended to operate asynchronously. The synthesis methodology is illustrated with two design examples.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Synthesis of artificial neural networks by transconductors only

Hardware implementation of artificial neural networks has been attracting great attention recently. In this work, the analog VLSI implementation of artificial neural networks by using only transconductors is presented. The signal flow graph approach is used in synthesis. The neural flow graph is defined. Synthesis of various neural network configurations by means of neural flow graph is described. The approach presented in this work is technology independent. This approach can be applied to new neural network topologies to be proposed or used with transconductors designed in future technologies.