Neural Network Optimization Research Articles

Utilization of genetic algorithm in tuning the hyper-parameters of hybrid NN-based side-slip angle estimators

This paper proposes a solution to enhance and compare different neural network (NN)-based side-slip angle estimators. The feed-forward neural networks (FFNNs), recurrent neural networks, long short-term memory units (LSTMs), and gated recurrent units are investigated. However, there is a lack in the selection criteria of the architectures’ hyper-parameters. Therefore, the genetic algorithm is integrated with the NN-based estimators to find the optimal hyper-parameters for the studied architectures. The tuned hyper-parameters in this work include the number of neurons, number of layers, activation function, optimizer type, and learning rate. The objective function of the optimization problem is minimizing the root-mean-square error (RMSE) on multiple testing data. The optimal models are further included in the design of a hybrid NN estimator with Kalman filter. In the hybrid estimators, the optimal NN estimators are used as virtual sensors to correct the prediction of the side-slip angle resulting from the mathematical lateral vehicle model. Eventually, the performance of the best selected model is evaluated in terms of different metrics; mean RMSE, mean error variance, mean training time, and mean estimation time. LSTMs are found to achieve the lowest mean RMSE while being tested on highly generalized data yielding the highest training and estimation time. However, FFNNs achieve the lowest RMSE while being tested on low generalized data and the lowest training and estimation time. Meanwhile, it is observed that the hybrid estimators achieved lower RMSE with great enhancement compared to the non-hybridized ones proving the effectiveness of the proposed approach and increasing the side-slip estimation generalization ability in unknown environments with high uncertainties, which are not covered by the training dataset for the NNs estimators.

Applying convolutional neural network and Nadam optimization in flower classification

Flowers have a variety of shapes, colors and structures, the images of which need to be classified using guided learning techniques. Several studies classify flowers using machine learning, but their accuracy performance is not good. The thing is, the flowers come in a variety of colors that can sometimes look similar to the background. Therefore, this study aims to classify flowers using a convolutional neural network (CNN) and measure its performance. The method used is mixed methods by collecting existing data from previous studies and connecting it with the realities in the field. The Kozłowski and Steinbrener models were used, while the image data was obtained from the Oxford17 and Oxford102 dataset with 17 and 102 flower types, respectively. The results show 60% and 84% accuracy of CNN using the scratch and transfer learning approach for the Oxford17 dataset. The Oxford102 dataset shows 42% and 64%, respectively, with CNN from baseline and transfer learning.

Optimization of Logistics Distribution Network based on Ant Colony Optimization Neural Network Algorithm

Optimization of Logistics Distribution Network based on Ant Colony Optimization Neural Network Algorithm

Neural Network Models for Ionospheric Electron Density Prediction at a Fixed Altitude Using Neural Architecture Search

AbstractSpecification and forecast of ionospheric parameters, such as ionospheric electron density (Ne), have been an important topic in space weather and ionospheric research. Neural networks (NNs) emerge as a powerful modeling tool for Ne prediction. However, heavy manual adjustments are time consuming to determine the optimal NN structures. In this work, we propose to use neural architecture search (NAS), an automatic machine learning method, to mitigate this problem. NAS aims to find the optimal network structure through the alternate optimization of the hyperparameters and the corresponding network parameters within a pre‐defined hyperparameter search space. A total of 16‐year data from Millstone Hill incoherent scatter radar (ISR) are used for the NN models. One single‐layer NN (SLNN) model and one deep NN (DNN) model are both trained with NAS, namely SLNN‐NAS and DNN‐NAS, for Ne prediction and compared with their manually tuned counterparts (SLNN and DNN) based on previous studies. Our results show that SLNN‐NAS and DNN‐NAS outperformed SLNN and DNN, respectively. These NN predictions of Ne daily variation patterns reveal a 27‐day mid‐latitude topside Ne variation, which cannot be reasonably represented by traditional empirical models developed using monthly averages. DNN‐NAS yields the best prediction accuracy measured by quantitative metrics and rankings of daily pattern prediction, especially with an improvement in mean absolute error more than 10% compared to the SLNN model. The limited improvement of NAS is likely due to the network complexity and the limitation of fully connected NN without the time histories of input parameters.

Optimization of electromagnetic layout for E-machine in electric vehicles: Achieving high performance, efficiency and NVH

The development of electric vehicles (EVs) presents the challenge of optimising electric machines to enhance efficiency, compactness, noise, vibration, harshness (NVH), and affordability, while reducing the dependence on heavy rare-earth materials to reduce the CO2 footprint of electric powertrains. This paper introduces a comprehensive optimization approach for the electromagnetic design of permanent magnet synchronous machines (PMSMs), employing a combination of Design of Experiment (DoE) and Robust Neural Networks (RNN). The optimization framework is utilised to comprehensively address the multiple objectives of an electric machine, including efficiency, noise, vibration, harshness (NVH), and cost. The integration of Artificial Intelligence (AI)-driven modelling has resulted in significant performance improvements, achieving up to 96% total efficiency over the entire load cycle with substantial NVH reductions of up to 20 dB, while reducing the magnet rare earth materials by 35% compared to the baseline. Furthermore, this methodology reduces the simulation time by up to 90%, demonstrating the potential of combining neural network optimization with conventional finite element simulation techniques. The validation of the AI-driven optimization approach with the measurement of the baseline and optimised electric machines for efficiency and vibration is demonstrated for correlation.

On the relevance of hyperparameter optimization of convolutional neural networks for retrieving spectral signals

On the relevance of hyperparameter optimization of convolutional neural networks for retrieving spectral signals

Damage identification of steel bridge based on data augmentation and adaptive optimization neural network

With the advancement of deep learning, data-driven structural damage identification (SDI) has shown considerable development. However, collecting vibration signals related to structural damage poses certain challenges, which can undermine the accuracy of the identification results produced by data-driven SDI methods in scenarios where data is scarce. This paper introduces an innovative approach to bridge SDI in a few-shot context by integrating an adaptive simulated annealing particle swarm optimization-convolutional neural network (ASAPSO-CNN) as the foundational framework, augmented by data enhancement techniques. Firstly, three specific types of noise are introduced to augment the source signals used for training. Subsequently, the source signals and augmented signals are recombined to construct a four-dimensional matrix as the input to the CNN, while defining the damage feature vector as the output. Secondly, a CNN is constructed to establish the mapping relationship between the input and output. Then, an adaptive fitness function is proposed that simultaneously considers the accuracy of SDI, model complexity, and training efficiency. The ASAPSO is employed to adaptively optimize the hyperparameters of the CNN. The proposed method is validated on an experimental model of a three-span continuous beam. It is compared with four other data-driven methods, demonstrating good effectiveness and robustness of SDI under cases of scarce data. Finally, the effectiveness of this SDI method is validated in a real-world case of a steel truss bridge.

Prediction of Surface Subsidence in Mining Areas Based on Ascending-Descending Orbits Small Baseline Subset InSAR and Neural Network Optimization Models.

Surface subsidence hazards in mining areas are common geological disasters involving issues such as vegetation degradation and ground collapse during the mining process, which also raise safety concerns. To address the accuracy issues of traditional prediction models and study methods for predicting subsidence in open-pit mining areas, this study first employed 91 scenes of Sentinel-1A ascending and descending orbits images to monitor long-term deformations of a phosphate mine in Anning City, Yunnan Province, southwestern China. It obtained annual average subsidence rates and cumulative surface deformation values for the study area. Subsequently, a two-dimensional deformation decomposition was conducted using a time-series registration interpolation method to determine the distribution of vertical and east-west deformations. Finally, three prediction models were employed: Back Propagation Neural Network (BPNN), BPNN optimized by Genetic Algorithm (GA-BP), and BPNN optimized by Artificial Bee Colony Algorithm (ABC-BP). These models were used to forecast six selected time series points. The results indicate that the BPNN model had Mean Absolute Errors (MAE) and Root Mean Squared Errors (RMSE) within 7.6 mm, while the GA-BP model errors were within 3.5 mm, and the ABC-BP model errors were within 3.7 mm. Both optimized models demonstrated significantly improved accuracy and good predictive capabilities.

Machine learning based analysis of entropy production and wall shear stress in blood flow through stenosed artery

The building up of cholesterol and various substances on the inner boundary of arterial walls forms a plaque (stenosis), which results in the narrowing of walls of arteries, and this exerts additional stress on the walls. Shear stress at the wall of arteries may cause the formation of plaque, plaque growth, and expansive arterial remodeling, and it may also result in an increased risk of plaque rupture. The purpose of this research work is to use machine learning approaches to analyze the wall shear stress and entropy generation through irreversible processes of blood flow in a porous stenosed artery within the context of magnetohydrodynamic consideration. As a way of illustrating blood’s non-Newtonian behavior, the Casson fluid model is used. A blood flow model in the context of mild stenosis has been developed, and it has subsequently been computationally addressed using an explicit finite difference method. The velocity, temperature, entropy generation, Bejan number, and wall shear stress are visualized for different values of pertinent variables. The result reflects the increased velocity in the stenosed region. A hybrid machine learning method using artificial neural network and particle swarm optimization is implemented to optimize the entropy generation and wall shear stress. The network is trained using Stochastic gradient–descent training procedure as well as Levenberg–Marquardt training procedure, and the outcomes of both training procedures are compared. Using this machine learning process, the minimum entropy production has been found for specific values of the Hartmann number, dimensionless temperature difference, and pulsatile component of pressure gradient. This machine learning-based optimization approach enhances the ability to forecast quantitative changes in affecting parameters, which will contribute to minimized damage to the arterial wall, improved blood flow efficiency and the delivery of necessary components to tissues.

Accurate identification of low-resistivity gas layer in tight sandstone gas reservoirs based on optimizable neural networks

In tight sandstone reservoirs, low resistivity gas reservoirs have a certain gas production capacity, but their reservoir resistivity value is less than 20 Ω m, or their ratio to water layer resistivity is less than 2. Therefore, the identification of low resistivity gas reservoirs is a prominent problem that needs to be resolved. In this paper, on the basis of traditional experimental analysis, NMR experiments and wettability experiments were conducted. The high bound water saturation and pore fractal dimension have a significant impact on reservoir resistivity. After various experimental analysis results were preprocessed by the image pool and data pool, an optimized neural network (ONN) classification model was established based on deep learning theory. The confusion matrix result (training datasets) was 87.13%, and the ROC area of low resistivity gas layers was close to 1, indicating good recognition performance for low resistivity gas layers. The model was used to identify the properties of the J4 well reservoir section. The identification result was consistent with the actual logging resistivity values (low resistivity) and testing conclusions (gas layer). This model can accurately identify low resistivity gas reservoirs in the research area, and has important guiding significance for identifying the reservoir properties of tight sandstone gas reservoirs in the Sichuan Basin. It can provide reference value for the exploration and development of similar gas reservoirs in other regions.

Design optimal neural network based on new LM training algorithm for solving 3D - PDEs

In this article, we design an optimal neural network based on new LM training algorithm. The traditional algorithm of LM required high memory, storage and computational overhead because of it required the updated of Hessian approximations in each iteration. The suggested design implemented to converts the original problem into a minimization problem using feed forward type to solve non-linear 3D - PDEs. Also, optimal design is obtained by computing the parameters of learning with highly precise. Examples are provided to portray the efficiency and applicability of this technique. Comparisons with other designs are also conducted to demonstrate the accuracy of the proposed design.

Research on Multimodal College English Teaching Model Based on Genetic Algorithm

Analyzing College English texts is essential for quantitatively evaluating their grammar, phrases, and words to enhance their use in writing, conversation, and other contexts. The precise and clear use of College English words, phrases, and sentences is essential to knowledge-based and foundational learning systems. Text data analytics run into problems with data amount, data diversity, data integration and interoperability. It is challenging to accomplish human-computer interaction in spoken College English communication and to assist students with corrections using the conventional methodology of teaching College English. Therefore, this paper proposed the Genetic Algorithm-based intelligent English course optimization system (GA-IECOS) to handle the scheduling above issue of college English classes and optimize college English teaching courses. The results demonstrate that the conventional BP neural network's local scheduling optimization issue may be resolved using the multidirectional mutation genetic BP neural network method. Subsequently, a mix of formative and summative assessments will be used to establish a couple of groups to evaluate the effectiveness using a control population and a trial group of a GA-IECOS for English language classes using a multidirectional mutation genetic algorithm and an optimization neural network. The results demonstrate that the GA-IECOS algorithm is more effective in the classroom and may greatly improve students' English performance

A BP Neural Network Product Design Optimization Model Based on Emotional Design and Sustainable Product Design

The objective of this study is to investigate the impact of emotional design and sustainable product design on user experience and satisfaction. Additionally, the study aims to optimize the combination efficiency and material selection methods of these two design approaches, extracting optimization parameters based on emotional and sustainable product design principles. By analyzing user satisfaction through questionnaires, various product indicators were examined. The study ultimately establishes a BP neural network optimization model by analyzing the design elements of different products and the satisfaction levels of various indicators. This BP neural network design optimization model, integrating emotional and sustainable design, demonstrates an accuracy rate of over 95% in predicting user satisfaction for different design schemes. Upon examining the emotional design and sustainable design parameter tables following data standardization, and analyzing various product models, it was concluded that the rate of change in each design parameter significantly impacts the satisfaction of each product indicator, highlighting the importance of these parameters in product design optimization.

Physical Delivery Network Optimization Based on Ant Colony Optimization Neural Network Algorithm

The development of modern logistics chains is not just simple cargo transportation, it has become a cross-integrated industry that integrates many emerging technologies such as IoT technology, intelligent transportation, cloud computing and mobile Internet. Based on the ant colony algorithm (ACA), this paper optimizes the physical delivery network of the optimized neural network algorithm, establishes a mathematical model for the constraints and optimization objectives in the optimization of the physical delivery path, and proposes some improvements to the ACA to improve the convergence of the algorithm. speed and global search ability, so as to use the improved algorithm to solve the physical delivery path optimization problem. Experiments show that the optimal distance of physical delivery path planning calculated by traditional ACA is 207.8544km, while the optimal distance of improved ACA path planning is 197.9879km. The performance of the improved ACA is improved by analyzing the results of solving typical examples.

Conventional and artificial intelligence based maximum power point tracking techniques for efficient solar power generation

AbstractThe increasing global need for renewable energy sources, driven by environmental concerns and the limited availability of traditional energy, highlights the significance of solar energy. However, weather fluctuations challenge the efficiency of solar systems, making maximum power point tracking (MPPT) systems crucial for optimal energy harvesting. This study compares ten MPPT approaches, including both conventional and artificial intelligence (AI)‐based techniques. These controllers were designed and implemented using MATLAB Simulink, and their performance was evaluated under real environmental conditions with fluctuating irradiance and temperature. The results demonstrate that conventional techniques, such as incremental conductance (INC), Perturb and Observe (P&O), Incremental conductance and Particle Swam Optimization (INC‐PSO), Fuzzy Logic Control and Particle Swam Optimization (FLC‐PSO), and Perturb and Observe and Particle Swam Optimization (P&O‐PSO), achieved accuracies of 94%, 97.6%, 98.9%, 98.7%, and 99.3% respectively. In contrast, AI‐based intelligent techniques, including Artificial Neural Network (ANN), Artificial Neural Fuzzy Interference System (ANFIS), Fuzzy Logic Control (FLC), Particle Swam Optimization (PSO), and Artificial Neural Network and Particle Swam Optimization (ANN‐PSO), outperform achieving higher accuracies of 97.8%, 99.9%, 98.9%, 99.2%, and 99%, respectively. Compared to available research, which often reports lower accuracies for conventional techniques, our study highlights the enhanced performance of AI‐based methods. This study provides a comprehensive comparative analysis, delivering critical analysis and practical guidance for engineers and researchers in selecting the most effective MPPT controller optimized to specific environmental conditions. By improving the efficiency and reliability of solar power systems, our research supports the advancement of sustainable energy solutions.

Multi-Objective Evolutionary Neural Architecture Search with Weight-Sharing Supernet

Deep neural networks have played a crucial role in the field of deep learning, achieving significant success in practical applications. The architecture of neural networks is key to their performance. In the past few years, these architectures have been manually designed by experts with rich domain knowledge. Additionally, the optimal neural network architecture can vary depending on specific tasks and data distributions. Neural Architecture Search (NAS) is a class of techniques aimed at automatically searching for and designing neural network architectures according to the given tasks and data. Specifically, evolutionary-computation-based NAS methods are known for their strong global search capability and have aroused widespread interest in recent years. Although evolutionary-computation-based NAS has achieved success in a wide range of research and applications, it still faces bottlenecks in training and evaluating a large number of individuals during optimization. In this study, we first devise a multi-objective evolutionary NAS framework based on a weight-sharing supernet to improve the search efficiency of traditional evolutionary-computation-based NAS. This framework combines the population optimization characteristic of evolutionary algorithms with the weight-sharing ideas in one-shot models. We then design a bi-population MOEA/D algorithm based on the proposed framework to effectively solve the NAS problem. By constructing two sub-populations with different optimization objectives, the algorithm can effectively explore network architectures of various sizes in complex search spaces. An inter-population communication mechanism further enhances the algorithm’s exploratory capability, enabling it to find network architectures with uniform distribution and high diversity. Finally, we conduct performance comparison experiments on image classification datasets of different scales and complexities. Experimental results demonstrate the effectiveness of the proposed multi-objective evolutionary NAS framework and the practicality and transferability of the introduced bi-population MOEA/D-based NAS method compared to existing state-of-the-art NAS methods.

Adaptation of the Neat Algorithm for Complex Problems with the Help of Quality Diversity Algorithms

The article discusses the essence of the NEAT (NeuroEvolution of Augmenting Topologies) algorithm in solving problems of neural network optimization and evolution of their topologies. An overview of the current state of use of NEAT and its adaptations in evolutionary computing research is given. The need for the Quality Diversity (QD) approach to increase the diversity and quality of solutions in complex problems is substantiated. The QD concept and its impact on the search for innovative solutions within diverse search spaces are described. The application of ViE-NEAT, which combines the advantages of survival of the fittest solutions with the principle of maintaining diversity, is described. The main aspects of ViE-NEAT are compared with the traditional NEAT, analyzing the advantages of using the survival algorithm in comparison with competitive methods. A detailed description of the MAP-Elites algorithm is given, which demonstrates an alternative approach to finding solutions by ensuring the diversity of the “illuminated” feature space, which can be integrated with NEAT to form a more diversified population of solutions. The main focus is on the methodology of integrating NEAT with MAP-Elites algorithms to create an adapted search strategy. Based on the basic principles of NEAT, the possibilities of its extension for effective solution of new problems that are not limited to traditional parametric spaces are determined. An analysis of the results demonstrating the efficiency of using the adapted NEAT algorithm in comparison with traditional approaches to the evolution of neural networks is presented.

Fault diagnosis of drivetrain systems based on vibration signals and convolutional neural networks

Accurate and fast fault diagnosis can improve the reliability of drivetrain systems. The fault diagnosis model based on convolutional neural networks is tricky because the accuracy and efficiency of diagnosis depend on the neural network parameters, such as layer size and number of layers. In this paper, we developed a data pre-processing technique to format vibrational data to be receptible to a neural network. First one will study the difference in performance by inputting a spectrogram of the vibrational data compared to the raw data itself. Second, the windowizing technique is used and its parameters are explored to train different configurations of convolutional neural networks. Finally, the optimal neural network parameters are obtained to improve the computational efficiency while the training model has the highest accuracy.

The modified Mohr-Coulomb model considering softening effect and intermediate principal stress

Based on the Mohr-Coulomb model, a modified Mohr-Coulomb model that can consider both softening effect and intermediate principal stress effect was formed by introducing plastic internal variables that characterize material damage and a modified stress Rhodes angle. The modified model is numerically integrated by implicit algorithm, the physical meaning of the model parameters of the modified model is clarified, and the multivariate analysis and sensitivity analysis of the model parameters are carried out. Combined with the results of the gypsum triaxial test, the model parameters of the modified model were determined by the equation solving and RBF neural network optimization method, and the validity of the model was verified. The research results showed that: The modified model established can consider both the material softening effect and the influence of the intermediate principal stress. Compared with the traditional Mohr-Coulomb model, the calculation results of the modified model are more consistent with the test results qualitatively and quantitatively. The comprehensive use of direct determination, equation solving and RBF neural network optimization methods can effectively determine the model parameters of the modified model, and the determination process is relatively simple. It is suggested that this combined method can be used for subsequent determination of more complex constitutive model parameters. Compared with the traditional Mohr-Coulomb model, the modified model only needs one more true-triaxial test to determine the model parameters. In addition, the experience of using the parameters of the traditional Mohr-Coulomb model in a large number of engineering practices can be used to improve the model, so the modified model has great potential for engineering applications.

A Fuzzy Wavelet Neural Network (FWNN) and Hybrid Optimization Machine Learning Technique for Traffic Flow Prediction

Traffic go with the flow forecasting is essential in urban planning and management, optimizing transportation structures and resource allocation. However, accurately predicting visitors glide is tough because of its inherent complexity, nonlinearity, and diverse uncertain factors. The trouble declaration underscores the issue in as it should be forecasting site visitors flow, mainly in urban environments characterized through dynamic and complex site visitor’s styles. In the existing paintings there are numerous traditional devices getting to know models used for visitors flow prediction, however those conventional strategies show off barriers in reaching excessive prediction accuracy. Therefore, the proposed work targets to put into effect hybrid optimization techniques for correct prediction in shipping machine. Here fuzzy wavelet neural community (FWNN) is used to address complicated nonlinear structures with uncertain conditions and hybrid optimization method called hybrid firefly and particle swarm optimization (HFO-PSO) which combines the exploration and exploitation talents of firefly and this fusion allows the version to capture intricate visitor’s styles efficiently and optimize the prediction technique, improving accuracy and efficiency. Moreover, the prediction performance of the proposed model is established and compared by means of the usage of distinct measures.