Training of Artificial Neural Network Using PSO With Novel Initialization Technique

The purpose of the research, the results of which are presented in this article, is to determine the possibility and evaluate the effectiveness of using a trained neural network in the diagnosis of ringworm. The article provides an analysis of the methods used for diagnosing dermatomycosis in veterinary practice. One of the actively developing areas at present is the use of artificial neural networks in the diagnosis of animal diseases. The authors have developed a method for diagnosing dermatophytosis using a trained neural network. To identify hair damaged by dermatophyte spores in cats, a trained artificial neural network YOLO v5 was used, based on the YOLO architecture (high-precision artificial neural network), which provides high accuracy and speed of object detection in images. Diagnostics was carried out in three stages. The first stage: the diagnosis of hair in cats damaged by dermatophyte spores was carried out using a trained artificial neural network. The second stage: microscopy by a veterinary specialist of the veterinary center. The third stage: comparison of the received data from the trained artificial neural network and veterinary specialists. Three comparative experiments were carried out on 20 depersonalized samples with different ratios from healthy and sick animals. As a result of testing the trichoscopy method using artificial neural networks for diagnosing spore-damaged hair dermatitis in cats, it was found that a trained artificial neural network of 60 studied samples diagnosed dermatophyte spore damage in 20 samples, a veterinarian - in 17. All positive results were confirmed by a mycological laboratory study. and identification of the pathogen. It has been established that the use of a trained artificial neural network increases the diagnostic efficiency by 15% and reduces the time to perform diagnostic microscopy by 60.3%. The application of the proposed method allows to reduce the time of microscopic examination, improve the accuracy of interpretation of the results, automate methods for identifying causative agents of ringworm in small animals and take timely measures to treat the animal.

Artificial neural network (ANN) has a wide variety of practice for the solution of problems in the area of data classification. Back propagation algorithm is a famous neural network (NN) traditional training approach. Hence, this classical training technique has many drawbacks like stuck in the local minima and maximum number of iterations required. Particle Swam Optimization (PSO) has been widely applied for the solutions of data classification problems. Population initialization is a vital factor in PSO algorithm, which considerably influences the diversity and convergence during the PSO's process. In this paper, the training of the ANN has been implemented with new initialization technique by using low discrepancies sequence, Torus termed as TO-PSO. In this paper, a detailed comparative performance analysis for the training of neural network is observed on nine benchmark data sets taken from UCI repository. The Results demonstrate that training of ANN with proposed initialization technique offer efficient and best substitute to traditional training approaches of the NN, which gives the solution of problems related to the data classification. Furthermore, the performance of TO-PSO has been compared with back propagation algorithm (BPA), standard PSO-NN and two other initialization approaches Sobol based PSO (SO-PSONN) and Halton based PSO (H-PSONN) for the training of ANN. The experimental results show that the proposed approach outperforms than BPA, traditional PSONN, SO-PSONN and H-PSONN in terms of converging speed and better accuracy Moreover, the outcomes of our work present a foresight that how the proposed initialization technique can be used as an efficient alternative to standard training approaches for the data classification problems.

PurposeThe paper aims to investigate the application of particle swarm optimisation and back propagation in weights optimisation and training of artificial neural networks within the mass appraisal industry and to compare the performance with standalone back propagation, genetic algorithm with back propagation and regression models.Design/methodology/approachThe study utilised linear regression modelling before the semi-log and log-log models with a sample of 3,242 single-family dwellings. This was followed by the hybrid systems in the selection of optimal attribute weights and training of the artificial neural networks. Also, the standalone back propagation algorithm was used for the network training, and finally, the performance of each model was evaluated using accuracy test statistics.FindingsThe study found that combining particle swarm optimisation with back propagation in global and local search for attribute weights enhances the predictive accuracy of artificial neural networks. This also enhances transparency of the process, because it shows relative importance of attributes.Research limitations/implicationsA robust assessment of the models’ predictive accuracy was inhibited by fewer accuracy test statistics found in the software. The research demonstrates the efficacy of combining two models in the assessment of property values.Originality/valueThis work demonstrated the practicability of combining particle swarm optimisation with back propagation algorithms in finding optimal weights and training of the artificial neural networks within the mass appraisal environment.

Comparing large number of metaheuristics for artificial neural networks training to predict water temperature in a natural river

BACKGROUND: Currently, artificial intelligence in the form of artificial neural networks is being actively implemented in a number of areas of our lives, including medicine. In particular, in otorhinolaryngology, artificial neural networks are used to analyze images obtained during endoscopic examinations of patients (e.g., videolaryngoscopy) [1–3]. The interpretation of laryngoscopic images often presents significant difficulties for practicing physicians, which reduces the frequency of detection of precancerous laryngeal diseases and contributes to the increase in the number of patients with stage III–IV laryngeal cancer [4, 5]. This underscores the significance of prompt performance and accurate interpretation of the findings of endoscopic examinations of patients with laryngeal disorders. Artificial neural networks can be employed to analyze the results of videolaryngoscopy, furnishing the physician with supplementary information that can enhance diagnostic accuracy and diminish the probability of error [6, 7]. AIM: The study aims to develop and train an artificial neural network for recognizing characteristic features of laryngeal neoplasms and variants of laryngeal normality. MATERIALS AND METHODS: The study was conducted under the grant of the Moscow Center for Innovative Technologies in Healthcare (grant No. 2112-1/22) entitled “Using Neural Networks (Artificial Intelligence Algorithms) for Control and Improving the Quality of Diagnosis and Treatment of Diseases of Laryngeal and Ear Structures through Digital Technologies”.The following methods were used during the course of the study: data collection for the creation of a photobank (dataset) of medical images obtained during videolaryngoscopy; data partitioning for the formation of datasets for individual nosologies and groups of diseases; the method of consilium; analysis of the accuracy of recognition and classification of digital endoscopic images; and training of classification neural networks. Consequently, a dataset comprising 1,471 laryngeal images in digital formats (JPEG, BMP) was assembled, labelled, and uploaded for the purpose of training the artificial neural network. Of the total number of images, 410 were classified as pertaining to laryngeal formation, while 1061 were classified as variants of normality. Subsequently, the neural network was trained and tested to identify the signs of normal and laryngeal masses. RESULTS: The results of the testing of the artificial neural network indicated the formation of an inaccuracy matrix, the calculation of the value of recognition accuracy, the calculation of the quality indicators of the model performance, and the construction of the ROC curve. The developed and trained artificial neural network demonstrated an accuracy of 86% in recognizing the signs of laryngeal masses and norms. CONCLUSIONS: This study demonstrates that a trained artificial neural network can successfully distinguish between signs of normal and laryngeal masses in endoscopic photographs. With further training of the neural network and achievement of high accuracy, this technology can be used in clinical practice as an assistant in the interpretation of laryngoscopic images and early diagnosis of laryngeal masses. It can also be employed to control and improve the quality of diagnosis and treatment of diseases of the throat, nose, and ears by primary care physicians.

This chapter aimed to evaluate heuristic approach performances for artificial neural networks (ANN) training. For this purpose, software that can perform ANN training application was developed using four different algorithms. First of all, training system was developed via back propagation (BP) algorithm, which is the most commonly used method for ANN training in the literature. Then, in order to compare the performance of this method with the heuristic methods, software that performs ANN training with genetic algorithm (GA), particle swarm optimization (PSO), and artificial immunity (AI) methods were designed. These designed software programs were tested on the breast cancer dataset taken from UCI (University of California, Irvine) database. When the test results were evaluated, it was seen that the most important difference between heuristic algorithms and BP algorithm occurred during the training period. When the training-test durations and performance rates were examined, the optimal algorithm for ANN training was determined as GA.

Training of Artificial Neural Networks (ANNs) have been improved over the years using meta heuristic algorithms that introduce randomness into the training method but they might be prone to falling into a local minima in a high-dimensional space and have low convergence rate with the iterative process. To cater for the inefficiencies of training such an ANN, a novel neural network is presented in this paper using the bio-inspired algorithm of the movement and mating of the mayflies. The proposed Mayfly algorithm is explored as a means to update weights and biases of the neural network. As compared to previous meta heuristic algorithms, the proposed approach finds the global minima cost at far less number of iterations and with higher accuracy. The network proposed, which is named Mayfly Algorithm based Neural Network (MFANN) consists of an input layer, a single hidden layer of 10 neurons and an output layer. Two University of California Irvine (UCI) database sample datasets have been used as benchmark for this study, namely the Banknote Authentication (BA) and the Cryotherapy, for which the training accuracy achieved is 99.2350% and 96.6102%, whereas the Testing accuracy is 99.1247% and 90% respectively. Comparative study with grey wolf optimization neural network (GWONN) and particle swarm optimization neural network (PSONN) reveal that the proposed MFANN achieves 1-2% better accuracy with training dataset and 2% better accuracy with testing dataset.

Comparing performances of backpropagation and genetic algorithms in the data classification

Sistem kimliklendirme ve modelleme için en yaygın kullanılan yapay zekâ tekniklerinden biri yapay sinir ağlarıdır. Yapay sinir ağları ile etkili sonuçlar elde etmek için etkili bir eğitim sürecine ihtiyaç duyulmaktadır. Meta-sezgisel algoritmalar pek çok gerçek dünya probleminin çözümünde başarılı bir şekilde kullanılmaktadır. Özellikle yapay sinir ağı eğitiminde, ağa ait parametrelerin optimizasyonu gerekmektedir. Son zamanlarda, bu amaçla meta-sezgisel algoritmalar kullanılmakta ve başarılı sonuçlar elde edilmektedir. Literatürde pek çok meta-sezgisel algoritma bulunmaktadır. Meta-sezgisel algoritmaların performansları problem türüne göre farklılık göstermektedir. Bu çalışma kapsamında ileri beslemeli yapay sinir ağının eğitiminde, yapay arı koloni algoritması, parçacık sürü algoritması, armoni arama, arı algoritması, çiçek tozlaşma algoritması ve guguk kuşu arama gibi popüler meta-sezgisel algoritmaların performansları değerlendirilmiştir. Uygulamalar için XOR, 2-bit parity ve 3-bit parity problemleri kullanılmıştır. Tüm problemler için elde edilen sonuçlar çözüm kalitesi ve yakınsama hızı açısından değerlendirilmiştir. Genel olarak ilgili problemlerin çözümü için meta-sezgisel algoritma tabanlı ileri yapay sinir ağı eğitiminin başarılı olduğu gözlemlenmiştir. En iyi sonuçlar ise yapay arı koloni algoritması ve guguk kuşu arama ile bulunmuştur.

<abstract><p>The training of artificial neural networks (ANNs) with rectified linear unit (ReLU) activation via gradient descent (GD) type optimization schemes is nowadays a common industrially relevant procedure. GD type optimization schemes can be regarded as temporal discretization methods for the gradient flow (GF) differential equations associated to the considered optimization problem and, in view of this, it seems to be a natural direction of research to <italic>first aim to develop a mathematical convergence theory for time-continuous GF differential equations</italic> and, thereafter, to aim to extend such a time-continuous convergence theory to implementable time-discrete GD type optimization methods. In this article we establish two basic results for GF differential equations in the training of fully-connected feedforward ANNs with one hidden layer and ReLU activation. In the first main result of this article we establish in the training of such ANNs under the assumption that the probability distribution of the input data of the considered supervised learning problem is absolutely continuous with a bounded density function that every GF differential equation admits for every initial value a solution which is also unique among a suitable class of solutions. In the second main result of this article we prove in the training of such ANNs under the assumption that the target function and the density function of the probability distribution of the input data are piecewise polynomial that every non-divergent GF trajectory converges with an appropriate rate of convergence to a critical point and that the risk of the non-divergent GF trajectory converges with rate 1 to the risk of the critical point. We establish this result by proving that the considered risk function is <italic>semialgebraic</italic> and, consequently, satisfies the <italic>Kurdyka-Łojasiewicz inequality</italic>, which allows us to show convergence of every non-divergent GF trajectory.</p></abstract>

Artificial Neural Network trained by Particle Swarm Optimization for non-linear channel equalization

Many problems in daily life exhibit nonlinear behavior. Therefore, it is important to solve nonlinear problems. These problems are complex and difficult due to their nonlinear nature. It is seen in the literature that different artificial intelligence techniques are used to solve these problems. One of the most important of these techniques is artificial neural networks. Obtaining successful results with an artificial neural network depends on its training process. In other words, it should be trained with a good training algorithm. Especially, metaheuristic algorithms are frequently used in artificial neural network training due to their advantages. In this study, for the first time, the performance of sixteen metaheuristic algorithms in artificial neural network training for the identification of nonlinear systems is analyzed. It is aimed to determine the most effective metaheuristic neural network training algorithms. The metaheuristic algorithms are examined in terms of solution quality and convergence speed. In the applications, six nonlinear systems are used. The mean-squared error (MSE) is utilized as the error metric. The best mean training error values obtained for six nonlinear systems were 3.5×10−4, 4.7×10−4, 5.6×10−5, 4.8×10−4, 5.2×10−4, and 2.4×10−3, respectively. In addition, the best mean test error values found for all systems were successful. When the results were examined, it was observed that biogeography-based optimization, moth–flame optimization, the artificial bee colony algorithm, teaching–learning-based optimization, and the multi-verse optimizer were generally more effective than other metaheuristic algorithms in the identification of nonlinear systems.

Background. There are a large number of neural networks that have their advantages and disadvantages, for example, simple, fast and easy to use single-stranded perceptrons are suitable for linear and linearized regression tasks, and more complicated neural networks are expendable in training and prediction time. Therefore, the problem arises for the development of fast and efficient algorithms for training artificial neural networks. An additional factor for researching new methods for training neural networks is finding the smallest training and prediction errors.Objective. The aim of the paper is to search and analyze the properties of the most effective method of training artificial neural networks using a combined approximation of the response surface. Another step is to perform computational experiments on proposed artificial neural networks and compare the results of experiments with known and developed methods.Methods. Analysis of known methods of combined approximation of the response surface was used. New algorithms for training neural networks, based on clustering of data using k-means method were developed. The algorithm with the smallest errors of artificial neural network learning and data prediction is chosen.Results. The results of research of different methods of training of artificial neural networks are given. Peculiarities of the methods of combined approximation of the response surface are analyzed. It is shown that the two methods of combined approximation of the response surface for training of artificial neural networks and prediction confirm the effectiveness of the proposed approach. Combined approximation algorithm is selected, which provides the lowest learning and forecasting errors.Conclusions. It was investigated that developed methods of combined approximation of the response surface allow training neural networks and predicting data with less error than when using autoregressive model with moving average, multilayer perceptron or artificial neural networks of models of geometric transformations without additional data processing.

Surface deflection is a phenomenon that causes fine wrinkles on the outer surfaces of sheet metal and deteriorates product external appearance. It is quantitatively defined as the difference between the section curve of the sheet and the ideal curve. In this study, using neural networks, a prediction model for surface deflection according to material properties was constructed and combined with a genetic algorithm; the combination of the material properties was studied to predict the minimum surface deflection. Because of the limited number of simulation data, neural networks were developed using several sampling methods such as central composite design, Latin hypercube sampling, and random sampling. In the training of the neural networks, the optimal hyper-parameter of the neural network was found automatically using Latin hypercube sampling. In conclusion, for prediction of surface deflection in rectangular embossing, neural networks made by central composite design showed the best performance. In addition, it was confirmed that the procedure of combining automatic training of a neural network and the genetic algorithm accurately predicted the set of material properties that generates the minimum surface deflection. Also, the quantity of surface deflection predicted by the neural network was very close to that predicted by finite element analysis.

Approaches presented today in the scientific literature suggest that there are no methodological solutions based on the training of artificial neural networks to predict the direction of industrial development, taking into account a set of factors—innovation, environmental friendliness, modernization and production growth. The aim of the study is to develop a predictive model of performance management of innovative industrial systems by building neural networks. The research methods were correlation analysis, training of neural networks (species—regression), extrapolation, and exponential smoothing. As a result of the research, the estimation efficiency technique of an innovative industrial system in a complex considering the criteria of technical modernization, development, innovative activity, and ecologization is developed; the prognostic neural network models allow to optimize the contribution of signs to the formation of target (set) values of indicators of efficiency for macro and micro-industrial systems that will allow to level a growth trajectory of industrial systems; the priority directions of their development are offered. The following conclusions: the efficiency of industrial systems is determined by the volume of sales of goods, innovative products and waste recycling, which allows to save resources; the results of forecasting depend significantly on the DataSet formulated. Although multilayer neural networks independently select important features, it is advisable to conduct a correlation analysis beforehand, which will provide a higher probability of building a high-quality predictive model. The novelty of the research lies in the development and testing of a unique methodology to assess the effectiveness of industrial systems: it is based on a multidimensional system approach (takes into account factors of innovation, environmental friendliness, modernization and production growth); it combines a number of methodological tools (correlation, ranking and weighting); it expands the method of effectiveness assessment in terms of the composition of variables (previously presented approaches are limited to the aspects considered).