Evolutionary Artificial Neural Networks Research Articles

Hedging policy using neural networks and its combination with heuristic algorithms case study: Dez Reservoir

In recent years, the use of various Artificial Intelligence (AI) methods, such as evolutionary computation, heuristic algorithms, artificial neural networks, and fuzzy theory calculations, has gained popularity in addressing water resources issues. These algorithms have shown great success in solving problems that traditional deterministic methods struggle with. This study focuses on optimizing Dez reservoir operation over a long-term period using a nonlinear loss function through an evolutionary artificial neural network algorithm. The outcomes of this approach are then contrasted with genetic exploration and harmony search algorithms, highlighting the strengths and weaknesses of each method. Ultimately, a combination of the evolutionary artificial neural network method and hedging models is employed for optimal reservoir management, with results compared to the previous approach. Results show the appropriate performance of combining hedging policy with artificial neural network and harmony search algorithm. This combination significantly reduces the vulnerability value with a slight decrease in reliability.

Cluster analysis and forecasting of viruses incidence growth curves: Application to SARS-CoV-2

The sanitary emergency caused by COVID-19 has compromised countries and generated a worldwide health and economic crisis. To provide support to the countries' responses, numerous lines of research have been developed. The spotlight was put on effectively and rapidly diagnosing and predicting the evolution of the pandemic, one of the most challenging problems of the past months. This work contributes to the existing literature by developing a two-step methodology to analyze the transmission rate, designing models applied to territories with similar pandemic behavior characteristics. Virus transmission is considered as bacterial growth curves to understand the spread of the virus and to make predictions about its future evolution. Hence, an analytical clustering procedure is first applied to create groups of locations where the virus transmission rate behaved similarly in the different outbreaks. A curve decomposition process based on an iterative polynomial process is then applied, obtaining meaningful forecasting features. Information of the territories belonging to the same cluster is merged to build models capable of simultaneously predicting the 14-day incidence in several locations using Evolutionary Artificial Neural Networks. The methodology is applied to Andalusia (Spain), although it is applicable to any region across the world. Individual models trained for a specific territory are carried out for comparison purposes. The results demonstrate that this methodology achieves statistically similar, or even better, performance for most of the locations. In addition to being extremely competitive, the main advantage of the proposal lies in its complexity cost reduction. The total number of parameters to be estimated is reduced up to 93.51% for the short term and 93.31% for the mid-term forecasting, respectively. Moreover, the number of required models is reduced by 73.53% and 58.82% for the short- and mid-term forecasting horizons.

An Evolutionary Artificial Neural Network approach for spatio-temporal wave height time series reconstruction

This paper proposes a novel methodology for recovering missing time series data, a crucial task for subsequent Machine Learning (ML) analyses. The methodology is specifically applied to Significant Wave Height (SWH) time series in the field of marine engineering. The proposed approach involves two phases. Firstly, the SWH time series for each buoy is independently reconstructed using three transfer function models: regression-based, correlation-based, and distance-based. The distance-based transfer function exhibits the best overall performance. Secondly, Evolutionary Artificial Neural Networks (EANNs) are utilised for the final recovery of each time series, using as inputs highly correlated buoys that have been intermediately recovered. The EANNs are evolved considering two metrics, the novel squared error relevance area, which balances the importance of extreme and around-mean values, and the well-known mean squared error. The study considers SWH time series data from 15 buoys in two coastal zones in the United States. The results demonstrate that the distance-based transfer function is generally the best transfer function, and that EANNs outperform a range of state-of-the-art ML techniques in 12 out of the 15 buoys, with a number of connections comparable to linear models. Furthermore, the proposed methodology outperforms the two most popular approaches for time series reconstruction, BRITS and SAITS, for all buoys except one. Therefore, the proposed methodology provides a promising approach, which may be applied to time series from other fields, such as wind or solar energy farms in the field of green energy.

Evolutionary artificial neural network for temperature control in a batch polymerization reactor

The integration of artificial intelligence techniques introduces fresh perspectives in the implementation of these methods. This paper presents the combination of neural networks and evolutionary strategies to create what is known as evolutionary artificial neural networks (EANNs). In the process, the excitation function of neurons was modified to allow asexual reproduction. As a result, neurons evolved and developed significantly. The technique of a batch polymerization reactor temperature controller to produce polymethylmethacrylate (PMMA) by free radicals was compared with two different controls, such as PID and GMC, demonstrating that artificial intelligence-based controllers can be applied. These controllers provide better results than conventional controllers without creating transfer functions to the control process represented.

Employing evolutionary artificial neural network in risk-adjusted monitoring of surgical performance

Employing evolutionary artificial neural network in risk-adjusted monitoring of surgical performance

One month in advance prediction of air temperature from Reanalysis data with eXplainable Artificial Intelligence techniques

In this paper we have tackled the problem of long-term air temperature prediction with eXplainable Artificial Intelligence (XAI) models. Specifically, we have evaluated the performance of an Artificial Neural Network (ANN) architecture with sigmoidal neurons in the hidden layer, trained by means of an evolutionary algorithm (Evolutionary ANNs, EANNs). This XAI model architecture (XAI-EANN) has been applied to the long-term air temperature prediction at different sub-regions of the South of the Iberian Peninsula. In this case, the average August air temperature has been predicted from ERA5 Reanalysis data variables, obtaining good predictions skills and explainable models in terms of the input climatological variables considered. A cluster analysis has been first carried out in terms of the average air temperature in the zone, in such a way that a number of sub-regions with different air temperature behaviour have been defined. The proposed XAI-EANN model architecture has been applied to each of the defined sub-regions, in order to find significant differences among them, which can be explained with the XAI-EANN models obtained. Finally, a comprehensive comparison against some state-of-the-art techniques has also been carried out, concluding that there are statistically significant differences in terms of accuracy in favour of the proposed XAI-EANN model, which also benefits from being an XAI model.

Constitutive modeling of Ta-rich particle reinforced Zr-based bulk metallic composites in the supercooled liquid region by using evolutionary artificial neural network

In this study, a series of in-situ Ta-rich particle reinforced Zr-based bulk metallic glass composites were successfully fabricated by arc-melting copper-mold spray casting. The effects of Ta content on the room temperature plasticity, compressive strength and thermoplastic formability were studied. (Zr55Cu30Al10Ni5)94Ta6 showed good comprehensive performance, and it was selected to systematically study the deformation behavior in the supercooled liquid region. Different from the strain softening after stress overshoot in bulk metallic glass, the composites showed work hardening in the late stage. Some classical constitutive models cannot accurately describe these phenomena. The back-propagation artificial neural network optimized by particle swarm optimization and genetic algorithm was used to establish the constitutive model. The particle-swarm-optimization back-propagation network with the optimal topology showed high accuracy and good generalization ability. The results predicted with this model were consistent with the experimental data, providing a powerful approach for describing the hot-deformation behavior of these Zr-based bulk metallic glass composites in the supercooled liquid region.

Pumps-as-Turbines’ (PaTs) performance prediction improvement using evolutionary artificial neural networks

Energy production from clean sources is mandatory to reduce pollutant emissions. Among different options for hidden hydropower potential exploitation, Pump-as-Turbine (PaT) represents a viable solution in pico- and micro-hydropower applications for its flexibility and low-cost. Pumps are widely available in the global market in terms of both sizes and spare parts. To date, there are several PaTs’ performance prediction models in the literature, but very few of them use optimization algorithms and only for specific and limited prediction goals. The present work proposes evolutionary Artificial Neural Networks (ANNs) based on JADE, which is a typology of differential evolution algorithm, to forecast Best Efficiency Point (BEP) and performance curves of a PaT starting from the pump operational data. In this model, JADE is employed as optimizer of basic ANNs to upgrade parameter values of the learning rate, weights, and biases. The accuracy of the proposed model is evaluated through experimental data available from the literature and compared to a basic ANN and two versions of the differential evolution algorithm. Results are also validated with experiments on a PaT showing that the proposed method can achieve an average R2-value of 0.97, which is 5% higher than the one obtained with a basic ANN.

COVID-19 contagion forecasting framework based on curve decomposition and evolutionary artificial neural networks: A case study in Andalusia, Spain

Many types of research have been carried out with the aim of combating the COVID-19 pandemic since the first outbreak was detected in Wuhan, China. Anticipating the evolution of an outbreak helps to devise suitable economic, social and health care strategies to mitigate the effects of the virus. For this reason, predicting the SARS-CoV-2 transmission rate has become one of the most important and challenging problems of the past months. In this paper, we apply a two-stage mid and long-term forecasting framework to the epidemic situation in eight districts of Andalusia, Spain. First, an analytical procedure is performed iteratively to fit polynomial curves to the cumulative curve of contagions. Then, the extracted information is used for estimating the parameters and structure of an evolutionary artificial neural network with hybrid architectures (i.e., with different basis functions for the hidden nodes) while considering single and simultaneous time horizon estimations. The results obtained demonstrate that including polynomial information extracted during the training stage significantly improves the mid- and long-term estimations in seven of the eight considered districts. The increase in average accuracy (for the joint mid- and long-term horizon forecasts) is 37.61% and 35.53% when considering the single and simultaneous forecast approaches, respectively.

Near real-time detection of blockages in the proximity of combined sewer overflows using evolutionary ANNs and statistical process control

Abstract Blockages are a major issue for wastewater utilities around the world, causing loss of service, environmental pollution, and significant clean-up costs. Increasing telemetry in combined sewer overflows (CSOs) provides the opportunity for near real-time data-driven modelling of wastewater networks. This paper presents a novel methodology, designed to detect blockages and other unusual events in the proximity of CSO chambers in near real-time. The methodology utilises an evolutionary artificial neural network (EANN) model for short-term CSO level predictions and statistical process control (SPC) techniques to analyse unusual level behaviour. The methodology was evaluated on historic blockage events from several CSOs in the UK and was demonstrated to detect blockage events quickly and reliably, with a low number of false alarms.

Simultaneous short-term significant wave height and energy flux prediction using zonal multi-task evolutionary artificial neural networks

The prediction of wave height and flux of energy is essential for most ocean engineering applications. To simultaneously predict both wave parameters, this paper presents a novel approach using short-term time prediction horizons (6h and 12h). Specifically, the methodology proposed presents a twofold simultaneity: 1) both parameters are predicted by a single model, applying the multi-task learning paradigm, and 2) the prediction tasks are tackled for several neighbouring ocean buoys with such single model by the development of a zonal strategy. Multi-Task Evolutionary Artificial Neural Network (MTEANN) models are applied to two different zones located in the United States, considering measurements collected by three buoys in each zone. Zonal MTEANN models have been compared in a two-phased procedure: 1) against the three individual MTEANN models specifically trained for each buoy of the zone, and 2) against some state-of-the-art regression techniques. Results achieved show that the proposed zonal methodology obtains not only better performance than the individual MTEANN models, but it also requires a lower number of connections. Besides, the zonal MTEANN methodology outperforms state-of-the-art regression techniques. Hence, the proposed approach results in an excellent method for predicting both significant wave height and flux of energy at short-term prediction time horizons.

Activating Silent Synapses in Sulfurized Indium Selenide for Neuromorphic Computing.

The transformation from silent to functional synapses is accompanied by the evolutionary process of human brain development and is essential to hardware implementation of the evolutionary artificial neural network but remains a challenge for mimicking silent to functional synapse activation. Here, we developed a simple approach to successfully realize activation of silent to functional synapses by controlled sulfurization of chemical vapor deposition-grown indium selenide crystals. The underlying mechanism is attributed to the migration of sulfur anions introduced by sulfurization. One of our most important findings is that the functional synaptic behaviors can be modulated by the degree of sulfurization and temperature. In addition, the essential synaptic behaviors including potentiation/depression, paired-pulse facilitation, and spike-rate-dependent plasticity are successfully implemented in the partially sulfurized functional synaptic device. The developed simple approach of introducing sulfur anions in layered selenide opens an effective new avenue to realize activation of silent synapses for application in evolutionary artificial neural networks.

Using evolutionary artificial neural networks in monitoring binary and polytomous logistic profiles

A statistical profile is a relationship between a quality characteristic (a response) and one or more explanatory variables to characterize quality of a process or a product. Monitoring profiles or checking the stability of profiles over time, has been extensively studied under the normal response variable, but it has paid a little attention to the profile with the non-normal response variable denoted by generalized linear models (GLM). Whereas, some of the potential applications of profile monitoring are cases where the response can be modelled using logistic profiles entailing binary, nominal and ordinal models. Also, most of existing control charts in this field have been developed by statistical approach and employing machine learning techniques have been rarely addressed in the related literature. Hence, to implement on-line process monitoring of logistic profiles, a novel artificial neural network (ANN) as a control chart with a heuristic training procedure is proposed in this paper. Performance of the proposed approach is investigated and compared using simulation studies in binary and polytomous models based on average run length (ARL) criterion. Simulation results revealed a good performance of the proposed approach. Nevertheless, to enhance the detection ability of the proposed approach more, the idea of combining run-rule which is a supplementary tool for making more sensitive control chart with final statistic is also implemented in this paper. Furthermore, a diagnostic method with machine learning schemes is employed to identify the shifted parameters in the profile. Results indicate the superior performance of the proposed approaches in most of the simulations. Finally, an example is used to illustrate the implementation of the proposed charting scheme.

An ANN-based ensemble model for change point estimation in control charts

Signaling in the control charts is usually followed by a substantial amount of delay, in which precise identification of the time when a change has occurred in a process simplifies the removal of change causes. This problem is referred to as change point (CP) estimation in the literature. This paper proposes a novel ensemble model to estimate CP under different processes’ changes in the phase II applications, known as ANNCP. It uses an evolutionary artificial neural network (ANN) as an underlying reasoning scheme to combine the predictions of multiple techniques and make the final decision. Specifically, a hybrid model of genetic algorithm (GA) and simulated annealing (SAN) with a new loss function is used to optimize the weights of ANN. The experimental results indicate that ANNCP can produce promising results under different conditions as compared with other state-of-the-art methods reported in the literature.

A Committee Evolutionary Neural Network for the Prediction of Combined Sewer Overflows

Combined Sewer Overflows (CSOs) are a major source of pollution and urban flooding, spilling untreated wastewater directly into water bodies and the surrounding environment. If overflows can be predicted sufficiently in advance, then techniques are available for mitigation. This paper presents a novel bi-model committee evolutionary artificial neural network (CEANN) designed to forecast water level in a CSO chamber from 15 min to 6 h ahead using inputs of past/current CSO level data, radar rainfall data and forecast forecasted rainfall data. The model is composed of two evolutionary artificial neural network (EANN) models. The two models are trained and optimised for wet and dry weather conditions respectively and their results combined into a single response using a non-linear weighted averaging approach. An evolutionary strategy algorithm is employed to automatically select the optimal artificial neural network (ANN) structure and parameter set, allowing the network to be tailored specifically for different CSO locations and forecast horizons without significant human input. The CEANN model was tested and evaluated on real level data from 4 CSOs located in Northern England and the results compared to three other ANN models. The results demonstrate that the CEANN model is superior in terms of accuracy for almost all forecast horizons considered. It is able to accurately forecast the dry weather and wet weather level, predicting the timing and magnitude of upcoming spill events, thus providing information that is of clear use to a wastewater utility.

Predicting Strength Ratio of Laminated Composite Material with Evolutionary Artificial Neural Network

In this paper, an alternative methodology to obtain the strength ratio for the laminated composite material is pre-sented. Traditionally, classical lamination theory and related fail-ure criteria are used to calculate the numerical value of strength ratio of laminated composite material under in-plane and out-of-plane loading from a knowledge of the material properties and its layup. In this study, to calculate the strength ratio, an alternative approach is proposed by using an artificial neural network, in which the genetic algorithm is proposed to optimize the search process at four different levels: the architecture, parameters, connections of the neural network, and active functions. The results of the present method are compared to those obtained via classical lamination theory and failure criteria. The results show that an artificial neural network is a feasible method to calculate the strength ratio concerning in-plane loading instead of classical lamination and associated failure theory.

An evolutionary neural network approach to machining process planning: A proof of concept

Developments in high degree-of-freedom(DOF) manufacturing processes such as 5-axis machining and additive manufacturing have greatly moderated the design constraint and brought unprecedented manufacturing capability for parts in complex geometry. The advancement in manufacturing processes, at the same time, leads to significant challenges for process planning due to the increasing decision complexity. A method is needed to enable full automated process planning for high DOF manufacturing processes in the foreseeable future. This work focuses on exploring an artificial neural network(ANN) based approach for machining process planning, specifically the toolpath planning for milling operations. The objective of this research is to construct a framework for automated machining process planning that leverages the advancement in ANN methodologies in an attempt to generate an optimized toolpath without any human logic input. In this proposed framework, the voxel model is used as part design and stock geometry representations. An evolving ANN method NeuralEvolution of Augmenting Topologies(NEAT) is applied as the solution algorithm. A prototype implementation of the proposed framework is created and experimented with reasonably simplified machining scenarios and basic part geometries. Initial experiments demonstrate optimistic results supporting the feasibility of creating such an ANN through an evolutionary method to accomplish specific manufacturing requirements on different geometries. The work also revealed that the geometric input is a critical factor for successfully training an ANN model. Further work is needed to encode the part design geometric information as input. Additionally, an improved evolutionary ANN algorithm needs to be created to accelerate the model training.

Concrete crack detection and 3D mapping by integrated convolutional neural networks architecture

This paper presents an image-based crack detection system, in which its architecture is modified to use deep convolutional neural networks in a feature extraction step and other classifiers in the classification step. In the classification step, classifiers including Support Vector machines (SVMs), Random Forest (RF) and Evolutionary Artificial Neural Network (EANN) are used as an alternative to a Softmax classifier and the performance of these classifiers are studied. The data set was created from various types of concrete structures using a standard digital camera and an unmanned aerial vehicle (UAV). The collected images are used in the crack detection system and in creating a 3D model of a sample concrete building using an image- based 3D photogrammetry technique. Then, the 3D model is used to create a mosaic image, in which the crack detection system was applied to create a global view of a crack density map. The map is then projected onto the 3D model to allow cracks to be located in the 3D world. A comparative study was conducted on the proposed crack detection system and the results prove that the combined architecture of CNN as a feature extractor and SVM as a classifier shows the best performance with the accuracy of 92.80. The results also show that the modified architecture by integrating CNN and other types of classifiers can improve a system performance, which is better than using the Softmax classifier.

Short- and long-term energy flux prediction using Multi-Task Evolutionary Artificial Neural Networks

This paper presents a novel approach to tackle simultaneously short- and long-term energy flux prediction (specifically, at 6h, 12h, 24h and 48h time horizons). The methodology proposed is based on the Multi-Task Learning paradigm in order to solve the four problems with a single model. We consider Multi-Task Evolutionary Artificial Neural Networks (MTEANN) with four outputs, one for each time prediction horizon. For this purpose, three buoys located at the Gulf of Alaska are considered. Measurements collected by these buoys are used to obtain the target values of energy flux, whereas, only reanalysis data are used as input values, allowing the applicability to other locations. The performance of three different basis functions (Sigmoidal Unit, Radial Basis Function and Product Unit) are compared against some popular state-of-the-art approaches such as Extreme Learning Machines and Support Vector Regressors. The results show that MTEANN methodology using Sigmoidal Units in the hidden layer and a linear output achieves the best performance. In this way, the multi-task methodology is an excellent and lower-complexity approach for energy flux prediction at both short- and long-term prediction time horizons. Furthermore, the results also confirm that reanalysis data is enough for describing well the problem tackled.

Evolutionary artificial neural networks for accurate solar radiation prediction

This paper evaluates the performance of different evolutionary neural network models in a problem of solar radiation prediction at Toledo, Spain. The prediction problem has been tackled exclusively from satellite-based measurements and variables, which avoids the use of data from ground stations or atmospheric soundings. Specifically, three types of neural computation approaches are considered: neural networks with sigmoid-based neurons, radial basis function units and product units. In all cases these neural computation algorithms are trained by means of evolutionary algorithms, leading to robust and accurate models for solar radiation prediction. The results obtained in the solar radiation estimation at the radiometric station of Toledo show an excellent performance of evolutionary neural networks tested. The structure sigmoid unit-product unit with evolutionary training has been shown as the best model among all tested in this paper, able to obtain an extremely accurate prediction of the solar radiation from satellite images data, and outperforming all other evolutionary neural networks tested, and alternative Machine Learning approaches such as Support Vector Regressors or Extreme Learning Machines.