Development Of Neural Networks Research Articles

Keratoconus Progression Determined at the First Visit: A Deep Learning Approach With Fusion of Imaging and Numerical Clinical Data.

Multiple clinical visits are necessary to determine progression of keratoconus before offering corneal cross-linking. The purpose of this study was to develop a neural network that can potentially predict progression during the initial visit using tomography images and other clinical risk factors. The neural network's development depended on data from 570 keratoconus eyes. During the initial visit, numerical risk factors and posterior elevation maps from Scheimpflug imaging were collected. Increase of steepest keratometry of 1 diopter during follow-up was used as the progression criterion. The data were partitioned into training, validation, and test sets. The first two were used for training, and the latter for performance statistics. The impact of individual risk factors and images was assessed using ablation studies and class activation maps. The most accurate prediction of progression during the initial visit was obtained by using a combination of MobileNet and a multilayer perceptron with an accuracy of 0.83. Using numerical risk factors alone resulted in an accuracy of 0.82. The use of only images had an accuracy of 0.77. The most influential risk factors in the ablation study were age and posterior elevation. The greatest activation in the class activation maps was seen at the highest posterior elevation where there was significant deviation from the best fit sphere. The neural network has exhibited good performance in predicting potential future progression during the initial visit. The developed neural network could be of clinical significance for keratoconus patients by identifying individuals at risk of progression.

Evolutionary neural networks for learning turbulence closure models with explicit expressions

Developing physical closure models with explicit expressions based on a given dataset is essential to science and engineering. For such symbolic regression tasks, biology-inspired evolutionary algorithms are most widely used. However, typical evolutionary algorithms do not utilize any structural information inherent in training data, which limits their performance in finding accurate model structures and coefficients. By combining one evolutionary algorithm, gene expression programing (GEP), with an artificial neural network (ANN) for symbolic regression, we propose a novel evolutionary neural network method, in which candidate expressions are specifically designed so that they can be transformed between the GEP and ANN structures during training iterations. By combining the GEP's global searching and the ANN's gradient optimization capabilities, efficient and robust convergence to accurate models can be achieved. In addition, sparsity-enhancing strategies have been introduced to improve the interpretability of the trained models. The present method has been tested for finding different physical laws and then applied to turbulence modeling problems with different configurations, showing advantages compared to the existing GEP and ANN methods.

Analysis of deep learning models for estimation of MPP and extraction of maximum power from hybrid PV-TEG: A step towards cleaner energy production

The Hybrid Photovoltaic-Thermoelectric generation system (PV-TEG) exhibits immense potential for effectively harnessing solar energy. However, the overall effectiveness of the hybrid PV-TEG system is hindered by the relatively low efficiency of both PV and thermoelectric generation (TEG). To overcome this limitation, this research investigates the application of an evolutionary Neural Network (NN)-based Maximum Power Point Tracking (MPPT) control technique. This proposed control technique utilizes a Growth Optimizer Algorithm (GO-algorithm) to optimize the weights and biases of a Deep Neural Network (DNN), enabling real-time tracking of the global maximum power point (GMM). Furthermore, to fortify the robustness of the MPPT control, a modified Fractional Order Proportional-Integral-Derivative (FOPID) controller is used whose gains are tuned using the GO-algorithm (GO-FOPID). Experimental evaluations conclusively demonstrate the efficacy of the evolutionary NN-based MPPT control technique in enhancing the generation efficiency of hybrid PV-TEG systems. The effectiveness of the proposed MPPT technique is validated through a comparative analysis with other MPPT techniques. The GO-DNN-based MPPT controller excels in its real-time global maxima tracking capability, exhibiting minimal power oscillations with average efficiency of 99.928% and an exceptionally low tracking time of less than 5ms. This superior performance is verified by comparative, simulation, quantitative, and statistical analyses, thereby highlighting the efficiency, tracking time, stability, and fault detection capabilities of the GO-DNN and GO-FOPID controllers across various practical scenarios.

Evolutionary neural network modeling of import substitution in the electronics industry of regions

Subject. This article focuses on the issues of evolutionary neural network modeling of import substitution capabilities and opportunities. Objectives. The article aims to study evolutionary neural network modeling in terms of identifying opportunities for import substitution in the electronics industry in the regions of Russia. The article also aims to identify the regions that are leaders in terms of the possibility of import substitution, and the regions that have prospects for the future development of the electronics industry within their territory. Results. The article presents the author-developed methodology for evolutionary neural network modeling of the possibility of import substitution in the electronics industry of the regions. Conclusions and Relevance. The results obtained can be useful for government agencies to plan the import substitution process in the electronics industry in regions mentioned. Investors can also use these results to choose the area of capital investment of their funds.

Optimization of a unileg thermoelectric generator by the combination of Taguchi method and evolutionary neural network for green power generation

The performance optimization of thermoelectric generators (TEGs) is crucial for efficiently converting waste heat to green energy. This study analyzes the performance and thermal stress of a unileg thermoelectric module by combining the Taguchi method, analysis of variance (ANOVA), and artificial neural network. The results of the Taguchi method indicate that the leg geometry exerts a negligible effect on output power (OP) and conversion efficiency but significantly affects the thermal stress (TS). The analysis of the objective function (maximum OP-to-maximum TS ratio, denoted by P/S) reveals the optimal configuration results in a P/S value of 0.01255 W‧MPa−1, accompanied by a maximum output power (2.9 W) and thermal stress (232.32 MPa). The maximum thermal stress of the rectangle TE leg (182.29 MPa) is less than that of the triangle (279.85 MPa) by 34.86 %. The Taguchi method, analysis of variance, and artificial neural network have the same sensitivity order on P/S with leg height > leg geometry > copper thickness > ceramic thickness. Furthermore, an evolutionary neural network (ENN) method with five-step optimization from 4000 datasets has been developed to improve machine learning prediction. The average error of P/S value and conversion efficiency from ENN is 1.61 % and 0.35 %, respectively. However, for the ENN predictions, the relative errors of P/S value and efficiency between predictions and simulations are 1.28 % and 0.19 %, respectively. In conclusion, this work has demonstrated the numerical optimization of the performance of unileg TEM for the first time. The use of ENN can predict the performance of TEM precisely, saving running time and markedly reducing computational costs. Moreover, the results are conducive to designing high-performance TEGs with long lifespans.

An enhanced structural developmental neural network with information saturation for continual unsupervised learning

In this paper, an enhanced structural developmental neural network with information saturation (ESDNNIS) is proposed for continual unsupervised learning. It improves SDNNIS in the following respects: (1) it makes a distinction between the developmental stages of neurons to better regulate the updating of neurons; (2) it adopts the forgetting mechanism to control the network size; (3) it changes the way coverage domain parameter is initialized and updated to divide clusters more accurately; and (4) it adds a parent–child relationship update mechanism to provide better robustness to sample input order. ESDNNIS inherits the unsupervised incremental learning and class segmentation capabilities of SDNNIS, and achieves better class incremental clustering for more complex datasets while using fewer neurons than SDNNIS. The experimental results show that the accuracy of clustering results of ESDNNIS is improved by 10.94% on average compared with SDNNIS, while the number of generated neurons is reduced by 71.93% on average.

Identification of partial differential equations from noisy data with integrated knowledge discovery and embedding using evolutionary neural networks

Identification of underlying partial differential equations (PDEs) for complex systems remains a formidable challenge. In the present study, a robust PDE identification method is proposed, demonstrating the ability to extract accurate governing equations under noisy conditions without prior knowledge. Specifically, the proposed method combines gene expression programming, one type of evolutionary algorithm capable of generating unseen terms based solely on basic operators and functional terms, with symbolic regression neural networks. These networks are designed to represent explicit functional expressions and optimize them with data gradients. In particular, the specifically designed neural networks can be easily transformed to physical constraints for the training data, embedding the discovered PDEs to further optimize the metadata used for iterative PDE identification. The proposed method has been tested in four canonical PDE cases, validating its effectiveness without preliminary information and confirming its suitability for practical applications across various noise levels.

Evaluation method for psychological resilience of athletes in high-intensity sports training based on evolutionary neural network

To enhance the psychological resilience of athletes, a method for evaluating the psychological resilience of High-intensity Interval Training (HIIT) athletes based on evolutionary neural networks is studied. From the six criteria of frustration coping, personal characteristics, self-promotion, self-regulation, internal protection and external protection, the evaluation index of psychological resilience of athletes in sports High-intensity Interval Training is selected; the audition indicators are qualitatively analyzed according to the principle of indicator selection, and the indicators that do not meet the requirements are eliminated; Cluster analysis and coefficient of variation analysis are used to carry out quantitative analysis on the remaining evaluation indicators after qualitative analysis; the indicators after quantitative analysis are improved, to build the assessment index system of psychological resilience of athletes in high-intensity sports training. The Back Propagation (BP) neural network is optimized by a genetic algorithm, and the evolutionary neural network is constructed. The index data set is input into the evolutionary neural network as a sample, and the index weight value is output through training. The evaluation result and corresponding evaluation grade are determined based on the index weight value and membership degree. The experimental results show that when the number of hidden layers is 3, the calculation of evaluation index weights is the best; The weight of personal traits obtained from the evaluation results is the highest (0.206), while the weight of external protection is the lowest (0.151), and the evaluation results are basically consistent with the expert results. The above results show that this method can accurately evaluate the psychological resilience of athletes and significantly enhance their psychological resilience.

Multiobjective visual evolutionary neural network and related convolutional neural network optimization

Biological visual systems intrinsically include a variety of direction-selective neurons capable of detecting changes in visual motion. Some of the neurons have been used to construct computational models for problem-specific engineering applications. However, it remains unclear how these neurons' visual response mechanisms can serve an interdisciplinary topic - visual evolutionary neural networks. In this study, we draw inspiration from swarm evolution and the fly's visual response and attention mechanisms to develop a fast multiobjective visual evolutionary neural network for solving multiobjective optimization problems, specifically focusing on convolutional neural network optimization. Our approach involves designing a multi-channel input visual neural network that outputs global and local learning rates for state transition, proposing a new population-like state update strategy to move current states toward the Pareto front as quickly as possible, and exploiting a point-to-plane distance model to control the size of the external archive set. The computational complexity of the evolutionary neural network mainly depends on the size of the external archive set and the number of fitness evaluations. Comparative experiments demonstrate that our network is an extremely competitive optimizer, effectively solving multiple kinds of benchmark examples. Especially, when optimizing the 8,248 parameters of a conventional convolutional neural network, our approach quickly achieves large-scale optimization.

Evolutionary Neural Networks for Option Pricing: Multi-Assets Option and Exotic Option

This paper presents a novel framework based on the evolutionary neural network to solve the generalized Black-Scholes equation arising in the financial market efficiently and accurately. We first employ evolutionary neural networks to parameterize the Partial Differential Equations (PDEs) involved in option pricing. This approach allows us to simplify the Black-Scholes PDEs and convert them into the corresponding Ordinary Differential Equations (ODEs). Thus we can use standard ODE solvers such as Euler’s method to solve the simplified ODE problem. The proposed framework is flexible and can handle various boundary conditions and terminal conditions, allowing customization based on specific market requirements and scenarios. Moreover, our method offers a reliable and deterministic solution methodology for the pricing framework, as it does not rely on stochastic training. Unlike other approaches that incorporate stochastic elements in their training process, our method eliminates the need for such training and provides consistent results. This deterministic nature enhances the reliability and stability of our approach, making it well-suited for real-world applica- tions in option pricing and financial markets. The experiments on multiple settings are carried out to illustrate the applicability and accuracy of the proposed framework.

Boundary delineation in transrectal ultrasound images for region of interest of prostate

Accurate and robust prostate segmentation in transrectal ultrasound (TRUS) images is of great interest for ultrasound-guided brachytherapy for prostate cancer. However, the current practice of manual segmentation is difficult, time-consuming, and prone to errors. To overcome these challenges, we developed an accurate prostate segmentation framework (A-ProSeg) for TRUS images. The proposed segmentation method includes three innovation steps: (1) acquiring the sequence of vertices by using an improved polygonal segment-based method with a small number of radiologist-defined seed points as prior points; (2) establishing an optimal machine learning-based method by using the improved evolutionary neural network; and (3) obtaining smooth contours of the prostate region of interest using the optimized machine learning-based method. The proposed method was evaluated on 266 patients who underwent prostate cancer brachytherapy. The proposed method achieved a high performance against the ground truth with a Dice similarity coefficient of 96.2% ± 2.4%, a Jaccard similarity coefficient of 94.4% ± 3.3%, and an accuracy of 95.7% ± 2.7%; these values are all higher than those obtained using state-of-the-art methods. A sensitivity evaluation on different noise levels demonstrated that our method achieved high robustness against changes in image quality. Meanwhile, an ablation study was performed, and the significance of all the key components of the proposed method was demonstrated.

An Evolutionary Neural Network Approach for Slopes Stability Assessment

A current big challenge for developed or developing countries is how to keep large-scale transportation infrastructure networks operational under all conditions. Network extensions and budgetary constraints for maintenance purposes are among the main factors that make transportation network management a non-trivial task. On the other hand, the high number of parameters affecting the stability condition of engineered slopes makes their assessment even more complex and difficult to accomplish. Aiming to help achieve the more efficient management of such an important element of modern society, a first attempt at the development of a classification system for rock and soil cuttings, as well as embankments based on visual features, was made in this paper using soft computing algorithms. The achieved results, although interesting, nevertheless have some important limitations to their successful use as auxiliary tools for transportation network management tasks. Accordingly, we carried out new experiments through the combination of modern optimization and soft computing algorithms. Thus, one of the main challenges to overcome is related to the selection of the best set of input features for a feedforward neural network for earthwork hazard category (EHC) identification. We applied a genetic algorithm (GA) for this purpose. Another challenging task is related to the asymmetric distribution of the data (since typically good conditions are much more common than bad ones). To address this question, three training sampling approaches were explored: no resampling, the synthetic minority oversampling technique (SMOTE), and oversampling. Some relevant observations were taken from the optimization process, namely, the identification of which variables are more frequently selected for EHC identification. After finding the most efficient models, a detailed sensitivity analysis was applied over the selected models, allowing us to measure the relative importance of each attribute in EHC identification.

Design and Simulation of a Neuroevolutionary Controller for a Quadcopter Drone

The problem addressed in the present paper is the design of a controller based on an evolutionary neural network for autonomous flight in quadrotor systems. The controller’s objective is to govern the quadcopter in such a way that it reaches a specific position, bearing on attitude limitations during flight and upon reaching a target. Given the complex nature of quadcopters, an appropriate neural network architecture and a training algorithm were designed to guide a quadcopter toward a target. The designed controller was implemented as a single multi-layer perceptron. On the basis of the quadcopter’s current state, the developed neurocontroller produces the correct rotor speed values, optimized in terms of both attitude-limitation compliance and speed. The neural network training was completed using a custom evolutionary algorithm whose design put particular emphasis on the cost function’s definition. The developed neurocontroller was tested in simulation to drive a quadcopter to autonomously follow a complex path. The obtained simulated results show that the neurocontroller manages to effortlessly follow several types of paths with adequate precision while maintaining low travel times.

An embedded Hamiltonian dynamic evolutionary neural network model for high-dimensional data recognition

Aiming at the limitation of Hamiltonian neural network in high-dimensional data processing with non-observable physical quantity system, an embedded Hamiltonian dynamic evolutionary neural network model (EHDN) is proposed in this paper. First, the dynamic physical quantities of high dimensional nonlinear systems are represented by constructing Hamiltonian functions. Then a symplectic integrator is designed to realize the time-varying evolution of network features based on Hamiltonian regular equation and variational numerical integration. Finally, the Hamiltonian dynamic evolutionary neural network is embedded in the convolutional neural network (CNN) to realize the Hamiltonian simulation evolution of convolutional features. Experimental results on two different datasets, CIFAR and Fashion-Mnist, show that compared with existing state-of-the-art (SOTA) methods, EHDN model can combine the advantages of Hamiltonian dynamics and convolutional network to improve the recognition accuracy and training stability in high-dimensional image classification tasks.

Multi-Objective Evolutionary Neural Network Optimization of Process Parameters for Double-Stepped Tube Hydroforming

Multi-Objective Evolutionary Neural Network Optimization of Process Parameters for Double-Stepped Tube Hydroforming

Evolutionary data driven modeling and tri-objective optimization for noisy BOF steel making data

Evolutionary data-driven modeling and optimization play a major role in generating meta models from real-time data. These surrogate models are applied effectively in various industrial operations and processes to predict a more accurate model from the nonlinear and noisy data. In this work, the data collected from a basic oxygen furnace of TATA steel are utilized in the modeling process by using evolutionary algorithms like evolutionary neural network (EvoNN), bi-objective genetic programming (BioGP), and evolutionary deep neural network (EvoDN2) to generate the meta models. For creating surrogates out In the current scenario of the Indian plants, reduction of phosphorus to an acceptable level, limiting the carbon and controlling the temperature are the basic needs in a basic oxygen furnace (BOF) to produce steels with a suitable composition. This work focused on three essential process parameters, temperature, carbon and phosphorus contents, and created intelligent models using 91 process variables of the operational process. The analysis began with a total of around 17000 operational observations and creating surrogate models out of them is a mammoth task, for which the data-driven evolutionary algorithms were some apt choices. Even there deep learning turned out to be essential and only the EvoDN2 algorithm performed at the expected level. Once the trained models are generated, optimization work was carried on three objectives simultaneously by using a constraint-based reference vector evolutionary algorithm (cRVEA). The optimized results were analyzed in multi-dimensional hyperspace, and their effectiveness in BOF steel making is presented in this work.

Data-driven green energy extraction: Machine learning-based MPPT control with efficient fault detection method for the hybrid PV-TEG system

The hybrid photovoltaic-thermoelectric generation system (PVTEG) gives two-fold benefits; firstly, it efficiently utilizes the available solar energy as it converts both solar irradiance and solar thermal energy into electricity, secondly, it enhances PV efficiency by reducing the PV module surface temperature. However, the efficiency of the hybrid PVTEG system is usually low because both PV and TEG are not highly efficient devices. Under changing environmental conditions, a well-designed maximum power point tracking (MPPT) controller can enhance the generation efficiency by 10%–15%. Therefore, this research explores the evolutionary Neural Network based MPPT control technique for the hybrid PVTEG systems. The snake optimizer optimally tuned weight and biases of the Multilayer perceptron neural network (MLPNN), which provides fast real-time global maxima (GM) tracking. Furthermore, to enhance the robustness of MPPT control, PID gains are tuned using the SO algorithm. Use of the snake optimizer based PID (SOPID) controller with the snake optimizer based neural network (SOANN) results in stable, accurate and fast MPPT under varying environmental conditions. The effectiveness of the proposed SOANN MPPT controller is validated by comparing it with PSOANN, RSANN and GWOANN. SOANN based MPPT controller provides very fast real-time global maxima (GM) tracking with negligible power oscillations. The SOANN controller extract optimal power with an average efficiency of 99.928% and tracking time of less than 5ms. Furthermore, an intelligent data driven based fault detection algorithm is proposed, which do not require any temperature or irradiance sensors reducing cost of the system. Comparative, simulation, quantitative, and statistical results second superior performance of SOANN controller in terms of efficiency, tracking time, stability and fault detection capability under various practical condition.

A structural developmental neural network with information saturation for continual unsupervised learning

AbstractIn this paper, we propose a structural developmental neural network to address the plasticity‐stability dilemma, computational inefficiency, and lack of prior knowledge in continual unsupervised learning. This model uses competitive learning rules and dynamic neurons with information saturation to achieve parameter adjustment and adaptive structure development. Dynamic neurons adjust the information saturation after winning the competition and use this parameter to modulate the neuron parameter adjustment and the division timing. By dividing to generate new neurons, the network not only keeps sensitive to novel features but also can subdivide classes learnt repeatedly. The dynamic neurons with information saturation and division mechanism can simulate the long short‐term memory of the human brain, which enables the network to continually learn new samples while maintaining the previous learning results. The parent‐child relationship between neurons arising from neuronal division enables the network to simulate the human cognitive process that gradually refines the perception of objects. By setting the clustering layer parameter, users can choose the desired degree of class subdivision. Experimental results on artificial and real‐world datasets demonstrate that the proposed model is feasible for unsupervised learning tasks in instance increment and class increment scenarios and outperforms prior structural developmental neural networks.

Inverter fault diagnosis based on Fourier transform and evolutionary neural network

The fault diagnosis of the inverter is fundamental to energy intelligence. Due to the complex characteristics of the inverter (e.g., high-dimensional decision and poor stability), it is challenging to solve the problem using traditional fault diagnosis methods. Recently, artificial intelligence (AI)-based approaches have emerged as the most promising methods. However, they often require to set hyperparameters manually, which hinders further AI-based applications in fault diagnosis of inverters. To fill the gap, we propose an inverter fault diagnosis method using fast Fourier Transform (FFT) and evolutionary neural network. This method combines the amplitude of low-frequency harmonic component of the three-phase inverter output current which is obtained by FFT and the average value in a period of three-phase inverter output current into the fault eigenvector. This method uses an evolutionary neural network trained by combining genetic algorithm (GA), ant colony optimization (ACO) algorithm and Back-propagation (BP) algorithm to realize fault diagnosis. This method can effectively resist noise interference and reduce the number of independent variables in the part of feature extraction, so that it can simplify the network model. In addition, this method can avoid the network training from trapping in local optima in the part of fault classification, with high accuracy and fast response speed. The experimental results show that the proposed algorithm and method of fault feature extraction can effectively detect and locate the insulated-gate bipolar transistor (IGBT) with open circuit (OC) fault in three-phase inverter, and can be applied to online monitoring.

Parameter estimation with the Markov Chain Monte Carlo method aided by evolutionary neural networks in a water hammer model

Parameter estimation with the Markov Chain Monte Carlo method aided by evolutionary neural networks in a water hammer model