Neural Network Optimization Research Articles

An optimized neural network with AdaHessian for cryptojacking attack prediction for Securing Crypto Exchange Operations of MEC applications

Bitcoin exchange security is crucial because of MEC's widespread use. Cryptojacking has compromised MEC app security and bitcoin exchange ecosystem functionality. This paper propose a cutting-edge neural network and AdaHessian optimization technique for cryptojacking prediction and defense. We provide a cutting-edge deep neural network (DNN) cryptojacking attack prediction approach employing pruning, post-training quantization, and AdaHessian optimization. To solve these problems, this paper apply pruning, post-training quantization, and AdaHessian optimization. A new framework for quick DNN training utilizing AdaHessian optimization can detect cryptojacking attempts with reduced computational cost. Pruning and post-training quantization improve the model for low-CPU on-edge devices. The proposed approach drastically decreases model parameters without affecting Cryptojacking attack prediction. The model has Recall 98.72%, Precision 98.91%, F1-Score 99.09%, MSE 0.0140, RMSE 0.0137, and MAE 0.0139. Our solution beats state-of-the-art approaches in precision, computational efficiency, and resource consumption, allowing more realistic, trustworthy, and cost-effective machine learning models. We address increasing cybersecurity issues holistically by completing the DNN optimization-security loop. Securing Crypto Exchange Operations delivers scalable and efficient Cryptojacking protection, improving machine learning, cybersecurity, and network management.

Validation study of Yunnan ethnic culture industry stock model under epidemic situation based on chaotic particle swarm optimization neural networks

The new crown pneumonia epidemic is raging, in the context of global integration, the scope of the impact of this sudden event spread around the world, the stock market has not been spared, the financial risk has increased dramatically compared with the past, the emergence of the epidemic has led to the spread of investor panic, March 2020, the U.S. S&P 500 index appeared in the four plunge, and led to the market trading meltdown, the world’s financial markets have had an extremely serious impact. The study of the impact of Xin Guan Pneumonia on the company’s stock returns is not only conducive to enriching the theoretical study of public health emergencies, but also conducive to improving the coping strategy, stabilizing the general economic market, and enhancing the public’s awareness of risk response. This paper compares the effect of the four intelligent algorithms of chaotic particle swarm algorithm, chaotic bee colony algorithm, chaotic fruit fly algorithm and chaotic ant colony algorithm combined with neural network on the prediction of the stock price trend of Yunnan national culture, and the study shows that the speed of convergence of the chaotic particle swarm optimization neural network and the speed of descent is better than that of the two models of chaotic fruit fly and chaotic bee colony, and the coefficients of decision of the chaotic particle swarm optimization neural network are higher than that of the other three models, and the errors are lower than the other three models. Indexes are lower than the other three models and have high accuracy in stock prediction of Yunnan ethnic culture, this finding emphasizes the potential of PSO-BP model to provide robust stock market prediction, which is important for both investors and policy makers in dealing with volatile market conditions.

Neural network and thermodynamic optimization of magnetized hybrid nanofluid dissipative radiative convective flow with energy activation

This article, motivated by hybrid magnetic coating manufacturing developments, utilizes a neural network-based computational program to study the dynamics of hybrid magnetic nanofluids with entropy generation. A new physico-chemo-mathematical model has been presented to simulate the hybrid magnetic nano-coating flow along a stretching surface to a porous medium with viscous heating. A Rosseland flux model is used for radiation heat transfer and Darcy’s model for the isotropic porous medium. The stretching sheet is porous, and wall suction or injection are possible. A robust neural network has been deployed to optimize the physical parameters controlling the transport characteristics of hybrid nanofluids. Specifically, two hybrid nanoparticle combinations are addressed, namely graphite oxide (GO)-molybdenum disulfide ( Mo S 2 ) and copper (Cu)-silicon dioxide ( Si O 2 ), both with engine oil as the base fluid. The dimensional boundary layer model is transformed via suitable scaling variables from a partial differential system into a dimensionless non-linear coupled ordinary differential system. The transformed boundary value problem is solved numerically with the BVP4C subroutine in the symbolic software MATLAB, which achieves exceptional accuracy. Validation with previous simpler studies is conducted and a good correlation is obtained. The neural network optimization analysis incorporates Bayesian regularization as the training algorithm. The Bejan entropy generation minimization (EGM) analysis shows that with increasing radiation parameter R d , both entropy generation rate and Bejan number are increased. Furthermore, an elevation in Brinkman number Br leads to an upsurge in entropy generation rate and a downtrend in the Bejan number. The numerical solution of the boundary value problem reveals that with an increment in nanoparticle solid volume fraction φ 2 , magnetic parameter M , inverse permeability parameter ϵ , surface injection parameter ( s < 0 ) , Eckert number Ec and radiation parameter R d and with a decrement in suction parameter ( s > 0 ) and Prandtl number Pr , there is a strong enhancement in temperature magnitude and thermal boundary layer thickness. With greater nanoparticle solid volume fraction φ 2 , magnetic parameter M , inverse permeability parameter ϵ , suction parameter s and a reduction in thermal buoyancy parameter λ , strong flow deceleration is induced, and momentum boundary layer thickness is increased. The skin friction coefficient is substantially boosted with lower values of magnetic parameter M , inverse permeability parameter ϵ , suction parameter s and higher values of thermal buoyancy parameter λ . There is a significant decrement also computed in Nusselt number with a greater radiation parameter R d . The simulations provide a good benchmark for future extensions that may consider non-Newtonian behavior.

The Investigation of Multiple Optimization Methods on Convolutional Neural Network

In this study, the optimization of a Convolutional Neural Network (CNN) was conducted using the Fruits 360 dataset, with a specific emphasis on the impacts of Spatial Transformer Network (STN) and Stochastic Gradient Descent (SGD) optimization methods. Firstly, a baseline CNN model is built, which achieves 97.84% accuracy with a loss of 0.0999 after 50 epochs. Then, the impact of integrating STN and SGD into CNN models separately is investigated. The addition of STN slightly increased the accuracy to 97.92%, reduced the loss to 0.0994, and decreased the validation accuracy. This result suggests that while STN enhances the model's generalization ability, it may slightly reduce the maximum accuracy achievable on the validation set. After SGD optimization, the verification accuracy is increased to 98.19%, the loss is reduced to 0.0537, and the verification accuracy is increased to 98.40%. These results highlight the effectiveness of SGD in fine-tuning model parameters, resulting in more accurate models and improved generalization capabilities. A comparative analysis of these methods highlights their respective advantages. The effectiveness of the STN is rooted in its capacity to improve model generalization and mitigate overfitting, which is particularly beneficial in situations that demand robustness against varied data sets. In contrast, SGD stands out for its ability to significantly improve model accuracy and reduce loss, making it a balanced choice for comprehensive model optimization. Future research directions include exploring these optimization techniques on various datasets and investigating the potential of combining STN and SGD to achieve higher performance in CNN models.

Improved Ionospheric Total Electron Content Maps over China Using Spatial Gridding Approach

Improved Ionospheric Total Electron Content Maps over China Using Spatial Gridding Approach

Non-volatile double-tunable vortex metalens design based on Sb2S3 using deep neural network and particle swarm optimization algorithm

A dual-tunable vortex metalens is proposed that leverages deep neural network (DNN) and particle swarm optimization algorithm (PSO), incorporating the use of the phase-change material Sb2S3, which enables simultaneous tunable of both the focal length of the focused vortex beam and the topological charge. The control of traditional vortex metalens relies on changing the polarisation state of the incident light; however, its control range is limited. We employed neural network algorithms to learn the complex mapping relationship between the unit structure and the phase, achieving a fast and accurate prediction of the vortex metalens's phase. Compared with traditional simulation methods, the application of this neural network significantly reduces the time consumption of the design process and avoids the need for re-simulation. Furthermore, our design transcends limitations associated with specific wavelengths and enables the rapid design of tunable metalens across the 1–1.5 μm wavelength range, offering flexibility for the development of dual-tunable metalens. Based on this scheme, we simulated vortex metalens with tunable focal length and topological charge, and the results showed that the simulation results were consistent with the theoretical predictions.

Intelligent optimization design of squeeze casting process parameters based on neural network and improved sparrow search algorithm

Squeeze casting process parameters are the key to squeeze casting production and to obtain excellent performance casts. To realize intelligent optimization design of process parameters under various requirements, this work presents a new intelligent optimization design framework for squeeze casting process parameters based on process data and integrating a two-stage intelligent integrated optimization. To adapt to diverse optimization applications of different process and target parameters, the backpropagation (BP) neural network and existing process data are utilized to intelligently establish the incompletely determined correlations between process parameters and squeeze cast quality or properties. An improved sparrow search algorithm (LCSSA) is developed and integrated to optimize both the aforementioned model structure and intelligently obtain the optimal solution, that is, the two stages of optimization. For the purpose of assessing the effect of each performance on the combination of process parameters, the information entropy weight approach is used. Various application cases and experiments have been conducted to evaluate the effectiveness of the suggested intelligent optimization design framework. It is indicated that the proposed method is practicable and can intelligently achieve ideal process parameters with high accuracy even based on small-scale data samples. The solving efficiency and optimization accuracy of LCSSA are superior than those of other intelligent optimization algorithms like the genetic algorithm (GA). The proposed framework outperforms the mixture of the BP neural network and other traditional optimization algorithms.

Modelling and optimizing hydraulic retention time in the biological aeration unit: Application of artificial neural network and particle swarm optimization

The biological aeration unit (BAU) is essential for removing organic and inorganic matter in the wastewater. Microorganisms present in the wastewater are able to remove organic and inorganic matter. The challenge is that the BAU consumes large quantities of energy during the process, due to the constant supply of air for the respiration of microorganisms. The standard hydraulic retention time (HRT) in the BAU is between 4 and 12 h per treatment cycle which is extensive, hence the high levels of energy consumption. The purpose of this study is to optimize the HRT in the BAU, by reducing the operation time of the air pumps/blowers, resulting in less energy consumption per treatment cycle. Particle swarm optimization (PSO) algorithm optimizes the HRT in the BAU. Variables in the optimization model are verified using the sensitivity analysis method. The results of the study show a reduction in HRT from 4 to 2.4954 h, which is a 37.6 % energy saving in the BAU. The biggest drivers in the HRT optimization model are energy consumption (61.3 %), airflow rate (36.8 %), and temperature (1.2 %). Therefore, decreasing airflow rate and increasing wastewater temperature reduces HRT in BAU.

Optimization and Improvement of Recurrent Neural Networks in Stock Price Prediction

Recurrent Neural Networks (RNNs) are a significant branch of deep learning, particularly suitable for handling data with temporal dependencies. In the financial domain, stock price prediction is a highly focused problem. Traditional stock price prediction methods, often based on linear models, fail to fully capture the nonlinear dynamics of stock prices. RNNs, with their memory capabilities, can capture the long-term dependencies of stock prices, hence offering great application potential in stock price prediction. However, RNNs also face challenges when processing stock price data, as stock prices are influenced by numerous factors and are highly complex and uncertain. Additionally, stock price data often contains noise and outliers, impacting the model’s predictive performance. To address these issues, incremental improvements in prediction accuracy and model generalization capabilities are achieved through optimizing the model structure and improving data preprocessing methods. As technology advances and algorithms continue to innovate, the application of RNNs in stock price prediction will become more widespread and in-depth.

Toward coordinated planning and hierarchical optimization control for highly redundant mobile manipulator

This paper represents a constraint planning and optimization control scheme for a highly redundant mobile manipulator considering a complex indoor environment. Compared with the traditional optimization solution of a redundant manipulator, infinity norm and slack variable are additionally introduced and leveraged by the optimization algorithm. The former takes into account the joint limits effectively by considering individual joint velocities and the latter relaxes the equality constraint by decreasing the infeasible solution area. By using derived kinematic equations, the tracking control problem is expressed as an optimization problem and converted into a new quadratic programming (QP) problem. To address the optimization problem, the two-timescale recurrent neural networks optimization scheme is proposed and tested with a 9 DOFs nonholonomic mobile-based manipulator. Additionally, the BI2RRT∗ path-planning algorithm incorporates path planning in the complex environment where different obstacles are positioned. To test and evaluate the proposed optimization scheme, both predefined and generated paths are tested in the Neurorobotics Platform (NRP) 22https://neurorobotics.net.which is open access and open source integrative simulation framework powered by Gazebo and developed by our team.

Deep neural networks optimization for resource-constrained environments: techniques and models

This paper aims to present a comprehensive review of advanced techniques and models with a specific focus on deep neural network (DNN) for resource-constrained environments (RCE). The paper contributes by highlighting the RCE devices, analyzing challenges, reviewing a broad range of optimization techniques and DNN models, and offering a comparative assessment. The findings provide potential optimization techniques and recommend a baseline model for future development. It encompasses a broad range of DNN optimization techniques, including network pruning, weight quantization, knowledge distillation, depthwise separable convolution, residual connections, factorization, dense connections, and compound scaling. Moreover, the review analyzes the established optimization models which utilizes the above optimization techniques. A comprehensive analysis is conducted for each technique and model, considering its specific attributes, usability, strengths, and limitations in the context of effective deployment in RCEs. The review also presents a comparative assessment of advanced DNN models’ deployment for image classification, employing key evaluation metrics such as accuracy and efficiency factors like memory and inference time. The article concludes with the finding that combining depthwise separable convolution, weight quantization, and pruning represents potential optimization techniques, while also recommending EfficientNetB1 as a baseline model for the future development of optimization models in RCE image classification.&lt;p&gt; &lt;/p&gt;

The construction of competency training mechanism model for tourism undergraduates based on grounded theory.

The motivation of this paper is to solve the problem of tourism majors' lack of theoretical knowledge and professional ability by building a capacity training mechanism model based on grounded theory. The goal of the study is to optimize the ability training strategies of tourism undergraduates and improve their professional quality and competitiveness. The paper adopts the ability training model based on grounded theory, and combines with Back Propagation (BP) neural network for optimization and evaluation. By comparing the performance of different ability training mechanisms, this paper explores the best training strategies to provide support and guidance for the training of tourism undergraduates. Firstly, the employment background of the current market is studied and analyzed. By sorting out the relevant documents of grounded theory and combining with the current training strategies of tourism majors, the students' personality characteristics and the basic principles of establishing models are integrated. The Back Propagation (BP) neural network is combined with the grounded theory. The data input of the student ability model is re-optimized. The undergraduate competency training mechanism model is constructed. The results show that when the number of iterations of the network model is 500.00, the evaluation accuracy of the competency training model based on BP neural network can reach 70.00%. At this time, the evaluation accuracy of competency training model based on content analysis method is only 55.00%. In addition, the results show that with the increase of model iterations, the recognition accuracy of the ability model based on grounded theory and the ability training mechanism of content analysis method is continuously improved. However, the ability evaluation model based on grounded theory has higher accuracy, and the accuracy of ability evaluation even exceeds 78.52% when the number of iterations of the network model is 600. Secondly, through the statistics and comparison of the grading results of students of different majors, it is found that the ability training mechanism based on grounded theory can improve the overall ability level of students more significantly. The research has important reference value for promoting the optimization and perfection of students' training strategies.

Optimization of Deep Neural Networks for Enhanced Efficiency in Small Scale Autonomous Vehicles

Optimization of Deep Neural Networks for Enhanced Efficiency in Small Scale Autonomous Vehicles

A Long Short-Term Memory with Recurrent Neural Network and Brownian Motion Butterfly Optimization for Employee Attrition Prediction

A Long Short-Term Memory with Recurrent Neural Network and Brownian Motion Butterfly Optimization for Employee Attrition Prediction

Novel Neural Network Models for Predicting Mental Health Outcomes in the U.S. Youth Population

Anxiety and depression are two of the most pressing mental health issues, particularly among young adults. Given the success of neural networks for predictive modeling, we developed novel neural network models for classifying anxiety and depression using Substance Abuse and Mental Health Services Administration’s Mental Health Client- Level Data (SAMHSA MH-CLD) on 382,174 young adults (15 to 24 years old). The SAMHSA MH-CLD included mental health and general background data collected in 2020 for individuals reporting to state-accredited hospital service centers across the United States. The neural networks were trained on 10% randomized k-folds of the dataset and tested on the remaining 90% for each fold. We found that neural network models predicted anxiety and depression with high accuracy (91.5% to 94.2% accuracy, 8.4% to 3.1% loss), outperforming conventional statistical models. Additionally, for all tested variable sets, our neural network model outperformed expectations for the average therapist consultation (46% to 50% accuracy), as reported previously. For the optimal neural network model with the highest accuracy, the variables most correlated with anxiety and depression were age, education, gender, race, employment, marital status, and stressor events, not accounting for redundant and minimally correlated variables. The effectiveness of our neural network model indicates that it can be implemented alongside therapists in clinical environments to improve psychiatric diagnosis among young adults.

Modeling Flood Susceptible Areas Using Deep Learning Techniques with Random Subspace: A Case Study of the Mae Chan Basin in Thailand

Flooding is a recurring global issue that leads to substantial loss of life and property damage. A crucial tool in managing and mitigating the impact of flooding is using flood hazard maps, which help identify high-risk areas and enable effective planning and management. This study presents a study on developing a predictive model to identify flood-prone areas in the Mae Chan Basin of Thailand using machine learning techniques, precisely the random sub-space ensemble method combined with a deep neural network (RS-DNN) and Nadam optimizer. The model was trained using 11 geographic information system (GIS) layers, including rainfall, elevation, slope, distance from the river, soil group, NDVI, road density, curvature, land use, flow accumulation, geology, and flood inventory data. Feature selection was carried out using the Gain Ratio method. The model was validated using accuracy, precision, ROC, and AUC metrics. Using the Wilcoxon signed-rank test, the effectiveness was compared to other machine learning algorithms, including random tree and support vector machines. The results showed that the RS-DNN model achieved a higher classification accuracy of 97% in both the training and testing datasets, compared to random tree (93%) and SVM (82%). The model's performance was also validated by its high AUC value of (0.99), compared to a random tree (0.93) and SVM (0.82) at a significance level of 0.05. In conclusion, the RS-DNN model is a highly accurate tool for identifying flood-prone areas, aiding in effective flood management and planning.

Deep-learning-based reconstruction of undersampled MRI to reduce scan times: a multicentre, retrospective, cohort study

The extended acquisition times required for MRI limit its availability in resource-constrained settings. Consequently, accelerating MRI by undersampling k-space data, which is necessary to reconstruct an image, has been a long-standing but important challenge. We aimed to develop a deep convolutional neural network (dCNN) optimisation method for MRI reconstruction and to reduce scan times and evaluate its effect on image quality and accuracy of oncological imaging biomarkers. In this multicentre, retrospective, cohort study, MRI data from patients with glioblastoma treated at Heidelberg University Hospital (775 patients and 775 examinations) and from the phase 2 CORE trial (260 patients, 1083 examinations, and 58 institutions) and the phase 3 CENTRIC trial (505 patients, 3147 examinations, and 139 institutions) were used to develop, train, and test dCNN for reconstructing MRI from highly undersampled single-coil k-space data with various acceleration rates (R=2, 4, 6, 8, 10, and 15). Independent testing was performed with MRIs from the phase 2/3 EORTC-26101 trial (528 patients with glioblastoma, 1974 examinations, and 32 institutions). The similarity between undersampled dCNN-reconstructed and original MRIs was quantified with various image quality metrics, including structural similarity index measure (SSIM) and the accuracy of undersampled dCNN-reconstructed MRI on downstream radiological assessment of imaging biomarkers in oncology (automated artificial intelligence-based quantification of tumour burden and treatment response) was performed in the EORTC-26101 test dataset. The public NYU Langone Health fastMRI brain test dataset (558 patients and 558 examinations) was used to validate the generalisability and robustness of the dCNN for reconstructing MRIs from available multi-coil (parallel imaging) k-space data. In the EORTC-26101 test dataset, the median SSIM of undersampled dCNN-reconstructed MRI ranged from 0·88 to 0·99 across different acceleration rates, with 0·92 (95% CI 0·92-0·93) for 10-times acceleration (R=10). The 10-times undersampled dCNN-reconstructed MRI yielded excellent agreement with original MRI when assessing volumes of contrast-enhancing tumour (median DICE for spatial agreement of 0·89 [95% CI 0·88 to 0·89]; median volume difference of 0·01 cm3 [95% CI 0·00 to 0·03] equalling 0·21%; p=0·0036 for equivalence) or non-enhancing tumour or oedema (median DICE of 0·94 [95% CI 0·94 to 0·95]; median volume difference of -0·79 cm3 [95% CI -0·87 to -0·72] equalling -1·77%; p=0·023 for equivalence) in the EORTC-26101 test dataset. Automated volumetric tumour response assessment in the EORTC-26101 test dataset yielded an identical median time to progression of 4·27 months (95% CI 4·14 to 4·57) when using 10-times-undersampled dCNN-reconstructed or original MRI (log-rank p=0·80) and agreement in the time to progression in 374 (95·2%) of 393 patients with data. The dCNN generalised well to the fastMRI brain dataset, with significant improvements in the median SSIM when using multi-coil compared with single-coil k-space data (p<0·0001). Deep-learning-based reconstruction of undersampled MRI allows for a substantial reduction of scan times, with a 10-times acceleration demonstrating excellent image quality while preserving the accuracy of derived imaging biomarkers for the assessment of oncological treatment response. Our developments are available as open source software and hold considerable promise for increasing the accessibility to MRI, pending further prospective validation. Deutsche Forschungsgemeinschaft (German Research Foundation) and an Else Kröner Clinician Scientist Endowed Professorship by the Else Kröner Fresenius Foundation.

Optimization of Nonlinear Convolutional Neural Networks based on Improved Chameleon Group Algorithm

In order to solve the most difficult problem of the architectural model established by CNN in solving specific problems, which results in parameter overflow and inefficient training, an optimization algorithm for nonlinear convolutional neural networks based on improved chameleon swarm algorithm is proposed. This article mainly introduces the use of Chameleon Swarm Optimization (PSO) algorithm to research the parameters of CNN architecture, solve them, and achieve the optimization of the optimization model.Although the number of parameters that need to be set up in CNN is very large, this method can find better testing space for Alexnet samples with 5 different images. In order to improve the performance of the improved pruning algorithms, two candidate pruning algorithms are also proposed. The experimental results show that compared with the traditional Alexnet model, the improved pruning method improves the image recognition ability of the Caffe primary parameter set from 1.3% to 5.7%. This method has wide applicability and can be applied to most neural networks which do not require any special functional modules of the Alexnet network model.

RETRACTED ARTICLE: Network security threat detection technology based on EPSO-BP algorithm

With the development of Internet technology, the large number of network nodes and dynamic structure makes network security detection more complex, which requires the use of a multi-layer feedforward neural network to build a security threat detection model to improve network security protection. Therefore, the entropy model is adopted to optimize the particle swarm algorithm to decode particles, and then the single-peak and multi-peak functions are used to test and compare the particle entropy and fitness values to optimize the weights and thresholds in the multi-layer feedforward neural network. Finally, Suspicious Network Event Recognition Dataset discovered by data mining is sampled and applied to the entropy model particle swarm optimization for training. The test results show that there are four functions for the optimal mean and standard deviation in this algorithm, with values of 5.712e − 02, 4.805e − 02, 4.914e − 01, 1.066e − 01, 1.577e − 01, 1.343e − 01, and 2.089e + 01, 5.926, respectively. Overall, the algorithm proposed in the study is the best. Finally, the detection rate of attack types is calculated. The multi-layer feedforward neural network algorithm is 83.80%, the particle swarm optimization neural network algorithm is 91.00%, and the entropy model particle swarm optimization algorithm is 95.00%. The experiment shows that the research model has high accuracy in detecting network security threats, which can provide technical support and theoretical assistance for network security protection.

Learning Accuracy with Particle Swarm Optimization for Music Genre Classification Using Recurrent Neural Networks

Deep learning has revolutionized many fields, but its success often depends on optimal selection hyperparameters, this research aims to compare two sets of learning rates, namely the learning set rates from previous research and rates optimized for Particle Swarm Optimization. Particle Swarm Optimization is learned by mimicking the collective foraging behavior of a swarm of particles, and repeatedly adjusting to improve performance. The results show that the level of Particle Swarm Optimization is better previous level, achieving the highest accuracy of 0.955 compared to the previous best accuracy level of 0.933. In particular, specific levels generated by Particle Swarm Optimization, for example, 0.00163064, achieving competitive accuracy of 0.942-0.945 with shorter computing time compared to the previous rate. These findings underscore the importance of choosing the right learning rate for optimizing the accuracy of Recurrent Neural Networks and demonstrating the potential of Particle Swarm Optimization to exceed existing research benchmarks. Future work will explore comparative analysis different optimization algorithms to obtain the learning rate and assess their computational efficiency. These further investigations promise to improve the performance optimization of Recurrent Neural Networks goes beyond the limitations of previous research.