Neural Network Optimization Research Articles

Structural Optimization Design of Vacuum Interrupter Based on Transient Electric Field Distribution

AbstractAs the core device of vacuum circuit breaker, the insulation performance of the interrupter is closely related to the internal structure. In this paper, the electric field distribution of arc interrupter under simulated lightning impulse overvoltage is studied. It is found that the instantaneous electric field intensity is at a high level and the electric field uniformity is poor under lightning impulse. In order to further improve the electric field distribution of the interrupter, this paper proposes a method based on the combination of neural network and particle swarm optimization (PSO) to optimize the important structural parameters of the interrupter. A neural network model was established with suspension shield length (L1), suspension shield radius (R1), end shield radius (R2), contact plate thickness (H1) and end shield length (H2) as inputs, and the maximum electric field strength and the uniformity of electric field on both sides of the static and dynamic contacts as outputs, respectively. The structure of the vacuum interrupter was optimized by PSO algorithm. The optimization results show that: When the structural parameters of the vacuum interrupter H1 are 4.6 mm, H2 is 20 mm, R1 is 3.6 mm, R2 is 3.4 mm and L1 is 110 mm, the electric field distribution of the interrupter is obviously improved, and the electric field distribution of the optimized structure can also be well improved under the power frequency operating voltage. Its electric field intensity between the electrode is significantly low. © 2024 Institute of Electrical Engineers of Japan and Wiley Periodicals LLC.

Superpixel-based principal feature clustering annotation method for dual-phase microstructure segmentation

Metallographic analysis is one of the most commonly used techniques by materials scientists for studying metal materials. The deep learning methods, which have been widely applied in metallographic images analysis, demonstrate excellent performance in this task. However, the optimization of deep learning models often relies on a substantial amount of accurately labeled samples for effective supervision. To address this issue, this paper proposes a novel automatic annotation method based on brightness and spatial distribution which is suitable for deep learning-based segmentation of optical dual-phase metallographic microstructure. The proposed automatic annotated method includes the superpixel segmentation, principal feature extraction, and clustering algorithm therefore it is referred as the superpixel-based principal feature clustering annotation (SPFCA) method. SPFCA employs discriminative criteria similar to those used by metallurgists to differentiate between metallographic structures. Furthermore, it can mitigate the occasional errors inherent in manual annotation, leading to improved performance compared to models trained with expert annotations. Experimental validation was conducted using four self-built datasets with different image qualities to test the performance of models from different perspective. Initially, hyperparameter optimization for the SPFCA method tailored to our dataset was performed. Subsequently, SPFCA was utilized to guide the optimization of the convolutional neural network employed for segmentation. The results demonstrate that the segmentation model optimized with SPFCA guidance achieved an F1 score of 0.9226 in the single dataset without the need for manual labeling, surpassing the segmentation models optimized with expert annotations.

Artificial Intelligence based Emission and Performance Prediction, and Optimization of HHO-Blended Gasoline SI Engine: A Sustainable Transition

In striving for sustainable alternatives to gasoline, Oxyhydrogen (HHO) has emerged as a promising substitute for Internal Combustion Engines (ICEs). HHO blends not only improve engine efficiency but also reduce harmful emissions. On-site, HHO utilization in the engine, eradicates low energy density and storage challenges. The current study combined cutting-edge machine learning (ML) techniques like Artificial Neural Network (ANN) and Gradient-based optimization to effectively utilize HHO with gasoline. Experimentation involved a single-cylinder spark ignition (SI) engine fueled by varying HHO-gasoline blends across different loads and speeds. Iterative tuning of the loss function led to the identification of the optimal architecture, denoted as 2HL-10N (2 hidden layers with 10 neurons each), with impressive correlation coefficients (0.99481 for training, 0.9781 for validation, 0.96914 for testing, and overall, 0.98819). Subsequently, ANN led Gradient-based optimization unveiled key performance metrics along with emissions. Upon implementing optimized conditions (HHO: 3.78 l/m, load: 100%, and 3465 rpm), notable enhancements were observed. The torque and efficiency increased by 11.8%, and 7.1%, respectively. Furthermore, brake-specific fuel consumption, carbon monoxide, and hydrocarbon emissions showed a reduction of 11.5%, 27.1%, and 36.6%, respectively. ANN based optimal engine operation revealed HHO as a potential replacement for conventional gasoline.

Battery Remaining Useful Life Estimation based on Particle Swarm Optimization-Neural Network

Determining the Remaining Useful Life (RUL) of a battery is essential for several purposes, including proactive maintenance planning, optimizing resource allocation, preventing unforeseen failures, improving safety, extending battery lifespan, and achieving accurate cost savings. Concerning that matter, this study proposed hybrid Particle Swarm Optimization–Neural Network (PSO-NN) for estimating battery RUL. In the evaluation of the proposed method, the effectiveness is assessed using the metrics of Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE). The dataset employed for this investigation comprises eight input parameters and one output variable, representing the battery RUL. In conducting an analysis, the performance of the PSO-NN model is compared with hybrid NN with Cultural Algorithm (CA-NN) and Harmony Search Algorithm (HSA-NN), as well as the standalone Autoregressive Integrated Moving Average (ARIMA). Upon examination of the findings, it becomes evident that the PSO-NN model outperforms the alternatives with an MAE of 2.7708 and an RMSE of 4.3468, significantly lower than HSA-NN (MAE: 22.0583, RMSE: 34.5154), CA-NN (MAE: 9.1189, RMSE: 22.4646), and ARIMA (MAE: 494.6275, RMSE: 584.3098). The PSO-NN also achieves the lowest maximum error of 104.7381 compared to 490.3125 for HSA-NN, 827.0163 for CA-NN, and 1,160.0000 for ARIMA. Additionally, the low two-tail probability values (P(T≤t)), all below the significance level of 0.05, indicate that the differences between PSO-NN and the other methods (HSA-NN, CA-NN, and ARIMA) are statistically significant. These results highlight the superior accuracy and robustness of the PSO-NN model in predicting battery RUL. This study contributes to the field by presenting the PSO-NN as a highly effective tool for accurate battery RUL estimation, as evidenced by its superior performance over alternative methods.

Implication of ICWFPSO as Optimization Neural Network Algorithm on Sales Forecasting System

Predictive systems play a crucial role in a company's operations and strategy by aiding in more informed and data-driven decision-making and more effective planning and budgeting. It is possible to develop an intelligent system to perform forecasting. Neural networks offer significant advantages in forecasting systems due to their flexible modeling capabilities. However, this algorithm's fundamental weakness is the slow convergence rate and being trapped in a local minimum. To overcome it, this research is conducted to optimize the NN algorithm using the ICWFPSO to produce a forecasting algorithm with high accuracy and faster execution time using real e-commerce sales data for the past 7 years. Algorithm performance testing tests the Mean Absolute Error (MAE) value of the forecasting system using three scenarios: the NN forecasting algorithm, the NN optimized with ICWFPSO on the weight value, and the same scheme. Still, the optimized value is the hyperparameter value. ICWPSO has been shown to enhance the performance of PSO by tuning the inertia weight dynamically, which helps balance exploration and exploitation during the optimization process. The best prediction result is obtained when optimizing the hyperparameters using the ICWFPSO optimization technique compared to using traditional NN or optimizing weight value with ICWFPSO with the MAE value of 245.32958984375, and the best performance is obtained at iterations below 100. Further, gradient-based optimization methods might be generally preferred for their efficiency and effectiveness in handling large-scale neural network training.

A Hybrid Deep Learning Model with Data Augmentation to Improve Tumor Classification Using MRI Images

Background: Cancer ranks second among the causes of mortality worldwide, following cardiovascular diseases. Brain cancer, in particular, has the lowest survival rate of any form of cancer. Brain tumors vary in their morphology, texture, and location, which determine their classification. The accurate diagnosis of tumors enables physicians to select the optimal treatment strategies and potentially prolong patients’ lives. Researchers who have implemented deep learning models for the diagnosis of diseases in recent years have largely focused on deep neural network optimization to enhance their performance. This involves implementing models with the best performance and incorporating various network architectures by configuring their hyperparameters. Methods: This paper presents a novel hybrid approach for improved brain tumor classification by combining CNNs and EfficientNetV2B3 for feature extraction, followed by K-nearest neighbors (KNN) for classification, which has been described as one of the simplest machine learning algorithms based on supervised learning techniques. The KNN algorithm assumes similarities between new cases and available cases and assigns new cases to the category that most closely resembles the available categories. Results: To evaluate the recommended method’s efficacy, two widely known benchmark MRI datasets were utilized in the experiments. The initial dataset consisted of 3064 MRI images depicting meningiomas, gliomas, and pituitary tumors. Images from two classes, consisting of healthy brains and brain tumors, were included in the second dataset, which was obtained from Kaggle. Conclusions: In order to enhance the performance even further, this study concatenated the CNN and EfficientNetV2B3’s flattened outputs before feeding them into the KNN classifier. The proposed framework was run on these two different datasets and demonstrated outstanding performance, with accuracy of 99.51% and 99.8%, respectively.

Optimization in Neural Networks with Mathematics: Efficiency in Architectures of Deep Learning

Optimization in Neural Networks with Mathematics: Efficiency in Architectures of Deep Learning

On the Existence of Minimizers in Shallow Residual ReLU Neural Network Optimization Landscapes

On the Existence of Minimizers in Shallow Residual ReLU Neural Network Optimization Landscapes

Tracking control strategy of tendon driven robotic arm under adaptive neural network

IntroductionWith the rapid optimization and evolution of various neural networks, the control problem of robotic arms in the area of automation control has gradually received more attention.MethodsTo improve the control performance of robotic arms under complex dynamic models, this study proposes an adaptive affective radial basis function network control strategy. Firstly, the kinematic and dynamic mathematical models of the tendon driven robotic arm are constructed. Then, by integrating the affective computing model and the radial basis function network, an adaptive affective radial basis function network control algorithm is constructed.Results and DiscussionThe research results indicate that the designed algorithm significantly outperforms the other two compared algorithms in terms of control accuracy and stability. In benchmark performance testing, the designed algorithm has a error accuracy of up to 0.97 and a steady state of up to 0.95. In the simulation results, the maximum torque change of the designed algorithm is only 3.8 Nm, which is much lower than other algorithms. In addition, the control error fluctuation range of this algorithm is between −0.001 and 0.001, almost close to zero error. This study provides a new optimization strategy for precise control of tendon driven robotic arms, and also opens up new avenues for the application of artificial intelligence technology in complex nonlinear system control.

Analysis and Prediction of Nonlinear Hysteretic Behaviors of Graphite Magnetorheological Grease at Different Temperatures.

Magnetorheological grease (MRG) is considered a promising alternative to magnetorheological fluids as a smart material because of its higher stability and less leakage. To enhance yield stresses in various applications, graphite is incorporated as an additive, resulting in graphite magnetorheological grease (GMRG). However, the nonlinear hysteresis properties of this new material and its prediction methods have not been investigated. Therefore, in this work, the nonlinear hysteretic properties of GMRG at different temperatures, magnetic fields, and frequencies are systematically investigated and compared with those of MRG. The results of rheological experiments show that graphite enhances the shear stress of GMRG in the hysteresis curve through its reinforcing effect on the magnetic chains. The strain hardening, elasticity, and viscosity of GMRG are enhanced, but an increase in temperature decreases this efficacy. A prediction model based on the particle swarm optimization neural network algorithm (PSO-BP) is also proposed to efficiently and accurately control the nonlinear behavior of GMRG in engineering. The hysteresis curves of GMRG under different external excitations are characterized and predicted by the particle swarm optimization neural network approach. The statistical results show that the PSO-BP-based hysteresis characteristic estimates have satisfactory accuracy. In the test data, the PSO-BP model demonstrates improvements in RMSE, MAE, and SMAPE by 19.5%, 20.6%, and 21.0%, respectively, compared to the conventional BP network, which maintains an R2 value exceeding 0.95. This method provides an efficient solution for researchers to carry out reliable performance prediction in work such as genetic engineering of materials and related engineering applications.

Automated neural network optimization for data-driven predictive models: an application to ROP in drilling

Automated neural network optimization for data-driven predictive models: an application to ROP in drilling

Experimental analysis and low-damage machining strategy for composite ultrasonic vibration-assisted grinding of silicon carbide based on DA-MLP-NSGA-II algorithm

At present, because of the lack of ultrasonic composite vibration assisted grinding mechanism, neural network optimization algorithm (NNOA) is used to optimize the processing results. In NNOA, multi-layer perceptron (MLP) neural network model and non-dominated sorting genetic algorithm-II (NSGA-II) are very efficient and accurate methods. In this paper, based on the measurement and analysis of the specific ultrasonic vibration device, the CUVAG experiments on silicon carbide (SiC) ceramic were carried out to investigate the influence of processing parameters on the grinding forces, the ground surface roughness and morphology, and the subsurface damage. Then, the brittle-ductile removal behavior of hard-and-brittle materials could be revealed according to the above analysis. After that, MLP model and NSGA-II were utilized to predict and optimize the processing results in CUVAG. The results show that the grinding forces are basically constant, the surface quality deteriorates, and the subsurface damage increases with increased axial vibration amplitude and workpiece infeed speed, but all fluctuate with enlarged wheel speed, and turns at the inflection point of brittle-ductile transition with increased elliptic vibration amplitude. The fitting goodness R2 of the established MLP neural network prediction model is between 0.94 and 0.975, and the process parameters calculated by the NSGA-II optimization algorithm are verified. With optimized processing parameters, the grinding forces are reduced by about 13 %, the surface roughness is reduced to Ra0.037 μm (by 29 %), and the depth of subsurface damage is reduced by 68 %.

Integrating handcrafted features with deep convolutional neural network and BWOA optimization for improved postmortem iris recognition system

Integrating handcrafted features with deep convolutional neural network and BWOA optimization for improved postmortem iris recognition system

Thermal energy analysis using artificial neural network and particle swarm optimization approach in partially ionized hyperbolic tangent material with ternary hybrid nanomaterials

This investigation holds significant pragmatic implications for endeavors aimed at curbing energy losses stemming from diverse factors. A neural network propelled by artificial intelligence, employing the Levenberg-Marquardt technique (ANN-LMM) has been devised for integrating ternary hybrid nanoparticles into a partially ionized hyperbolic tangent liquid flowing over an extended melting surface (PIHTL-SMS). The substance motion equivalence is delineated, considering the rotational outcome. The heat energy is formulated by amalgamating viscous intemperance and Joule heat contributions. To streamline complexity, the resulting PDEs are transmuted into a series of ordinary differential equations (ODEs) through resemblance transformations. A reference dataset for ANN-LMM is produced encompassing diverse significant model permutations and pretending situations utilizing the Lobatto III-A statistical technique. This reference data undergoes verification, evaluation and training procedures to refine the estimated explanation towards achieving anticipated outcomes. The precision, constancy, capability and resilience of ANN-LMM are substantiated concluded mean squared error (MSE)-based fitness curves, error histograms, regression plots and absolute error evaluations. A relative examination elucidates the correctness of the suggested solver, exhibiting entire errors within the array of 10-10 to 10-06 for all significant constraints. Resulting differential equations are also solved using particle swarm optimization (PSO) approach. In PSO several parameters are optimized to enhance the performance of the algorithm. Optimizing these parameters help to improve the effectiveness and efficiency of the PSO algorithm for given problem. PSO converges quickly to optimal or near-optimal solutions, making it efficient for problems with large domain. Several pivotal graphs are constructed to illustrate the impact of emergent constraints on fluid temperature and velocity profiles. The outcomes underscore the numerical technique as a potent instrument for tackling the intricate conjoined ODEs system prevalent in fluid mechanism and allied intemperance presentations in technology. Additionally, improvements in the Forchheimer constraint and the Weissenberg number are deemed imperative for regulating fluid velocity. Unlike prior research that mostly concentrated on single or binary nanofluids, this work presents the integration of ternary hybrid nanoparticles into a partly ionized hyperbolic tangent liquid, a unique technique. Improved accuracy and processing efficiency are also provided by using an ANN-LMM neural network to solve the complicated transformed ODEs. Comparison of ANN, PSO results and existing results are done which shows validity of the current analysis. This work is unique in that it offers a deeper understanding of fluid behavior at advanced thermal settings by including emergent restrictions, viscous dissipation, and Joule heating.

Simplified Knowledge Distillation for Deep Neural Networks Bridging the Performance Gap with a Novel Teacher–Student Architecture

The rapid evolution of deep learning has led to significant achievements in computer vision, primarily driven by complex convolutional neural networks (CNNs). However, the increasing depth and parameter count of these networks often result in overfitting and elevated computational demands. Knowledge distillation (KD) has emerged as a promising technique to address these issues by transferring knowledge from a large, well-trained teacher model to a more compact student model. This paper introduces a novel knowledge distillation method that simplifies the distillation process and narrows the performance gap between teacher and student models without relying on intricate knowledge representations. Our approach leverages a unique teacher network architecture designed to enhance the efficiency and effectiveness of knowledge transfer. Additionally, we introduce a streamlined teacher network architecture that transfers knowledge effectively through a simplified distillation process, enabling the student model to achieve high accuracy with reduced computational demands. Comprehensive experiments conducted on the CIFAR-10 dataset demonstrate that our proposed model achieves superior performance compared to traditional KD methods and established architectures such as ResNet and VGG networks. The proposed method not only maintains high accuracy but also significantly reduces training and validation losses. Key findings highlight the optimal hyperparameter settings (temperature T = 15.0 and smoothing factor α = 0.7), which yield the highest validation accuracy and lowest loss values. This research contributes to the theoretical and practical advancements in knowledge distillation, providing a robust framework for future applications and research in neural network compression and optimization. The simplicity and efficiency of our approach pave the way for more accessible and scalable solutions in deep learning model deployment.

A hybrid algorithm for photovoltaic maximum power point tracking using Artificial Neural Network and Kinetic Gas Molecular Optimization

The quest for efficient photovoltaic (PV) energy conversion systems has led to the development of Maximum Power Point Tracking (MPPT) algorithms. In this paper, we propose a novel hybrid approach that combines the power of Artificial Neural Network (ANN) with the optimization capability of Kinetic Gas Molecular Optimization (KGMO) for MPPT. We present a high-level outline of the proposed algorithm along with example MATLAB code snippets for each step, highlighting its potential for improving PV system performance. This paper proposes a metaheuristic optimized multilayer feed‐forward artificial neural network (ANN) controller to extract the maximum power from available solar energy. Firstly, to improve the maximum power point (MPP) delivered by PV arrays and to overcome the drawbacks in the conventional MPPT method under irradiation variation, a hybrid MPPT controller is designed, in which the input parameters include the PV array voltage and current. The output parameter is the duty cycle of the DC/DC boost converter. The proposed approach abbreviated as ANN-KGMO MPPT controller is based on the Kinetic Gas Molecular Optimization (KGMO) Algorithm which is useful to train the developed ANN and to evolve the connection weights and biases to get the optimal values of duty cycle converter corresponding to the MPP of a PV array. Finally, the performance of the proposed control system is confirmed by simulation tests on a 2 kW PV system. In addition, the performance of an ANN-KGMO -based MPPT controller is also compared to the conventional perturb and observe (P&O) method. To analyze the results, simulations are performed by using MATLAB software.

Methodology for hyperparameter tuning of deep neural networks for efficient and accurate molecular property prediction

This paper presents a methodology of hyperparameter optimization (HPO) of deep neural networks for molecular property prediction (MPP). Most prior applications of deep learning to MPP have paid only limited attention to HPO, thus resulting in suboptimal values of predicted properties. To improve the efficiency and accuracy of deep learning models for MPP, we must optimize as many hyperparameters as possible and choose a software platform to enable the parallel execution of HPO. We compare the random search, Bayesian optimization, and hyperband algorithms, together with the Bayesian-hyperband combination within the software packages of KerasTuner and Optuna for HPO. We conclude that the hyperband algorithm, which has not been used in previous MPP studies, is most computationally efficient; it gives MPP results that are optimal or nearly optimal in terms of prediction accuracy. Based on our case studies, we recommend the use of the Python library KerasTuner for HPO.

INTELLIGENT MONITORING OF THE PHYSIOLOGICAL STATE OF AGRICULTURAL PRODUCTS USING UAV

The article discusses the technology for automated neural network monitoring of the vineyard’s physiological condition. Images of leaves, obtained using an unmanned aerial vehicle (UAV), are the main indicator of the physiological vineyard’s condition. The proposed solution is based on the integrated use of convolutional neural network method (CNN) and machine vision technologies. To determine the optimal neural network (NN) model, a variant analysis was carried out. In accordance with its results, the YOLOv7 model was chosen, which satisfies the introduced time limit and provides the required detection quality. The training of the YOLOv7 neural network was implemented in the Python environment using the PyTorch framework and the OpenCV computer vision library. The dataset consisting of 6320 images of grape leaves (including healthy and diseased ones) has been used for neural network training. The obtained results showed that the detection accuracy is at least 91%. Visualization of monitoring results has been carried out using heatmap, allowing to obtain information about vineyard physiological condition in dynamics. The proposed mathematical model allows to calculate the monitored vineyard’s area made by one complex per day. The obtained results showed that effective monitoring area using one DJI Phantom 4 UAV per day is 2.5 hectares.

Water Supply Pipeline Failure Evaluation Model Based on Particle Swarm Optimization Neural Network

The degradation and failure of the urban water supply network may lead to serious safety hazards, including pipe breaks, water supply interruptions, water resource losses, and contaminant intrusions. The risk evaluation of water supply pipeline failure in a distribution network is a challenging task, because most of the available data cannot fully reflect pipeline failure events and many of the mechanisms still cannot be fully understood. Therefore, a predictive model is urgently needed to assess pipeline failure risk based on available data. In this paper, based on the traditional risk assessment theory, seven main factors affecting pipeline failure are selected and scored, and then a pipeline failure model is established by using the particle swarm optimization (PSO) neural network. The model uses the neural network training of historical data to evaluate the failure of the water supply pipeline, and the PSO is used to optimize the neural network to effectively improve the training time and accuracy. The model error and correlation coefficient are 0.003 and 0.987, respectively. The proposed model can be used as a powerful support tool to assist infrastructure managers and pipeline maintainers in their plans and decision-making.

Bayesian mesh optimization for graph neural networks to enhance engineering performance prediction

Abstract In engineering design, surrogate models are widely employed to reduce the computational costs of simulations by approximating design variables and geometric parameters from computer-aided design (CAD) models. However, traditional surrogate models often lose critical information when simplified to lower dimensions, and face challenges in handling the complexity of 3D shapes, especially in industrial datasets. To address these limitations, we propose a Bayesian graph neural network (GNN) framework that directly learns geometric features from CAD mesh representations for accurate engineering performance prediction. Our framework leverages Bayesian optimization (BO) to dynamically determine the optimal mesh element size, significantly improving model accuracy while balancing computational efficiency. This approach addresses the challenge of mesh quality by optimizing mesh resolution, ensuring that critical geometric features are preserved in 3D deep-learning-based surrogate models. Unlike existing models that rely on static or predefined mesh resolutions, our method adapts mesh size based on the task, making it highly flexible for a wide range of engineering applications. Experimental results demonstrate that mesh quality directly impacts prediction accuracy, and the proposed BO-EI GNN model outperforms state-of-the-art models such as 3D CNN, SubdivNet, GCN, and GNN in predicting mass, rim stiffness, and disk stiffness. Our method also significantly reduces the computational costs compared to traditional optimization techniques. The proposed framework shows promising potential for application in finite element analysis (FEA) and other mesh-based simulations, enhancing the utility of surrogate models across various engineering fields.