Weights Of Neural Network Research Articles

Compressive strength of nano concrete materials under elevated temperatures using machine learning

In this study, four Artificial intelligence (AI) - based machine learning models were developed to estimate the Residual compressive strength (RCS) value of concrete supported with nano additives of Nanocarbon tubes (NCTs) and Nano alumina (NAl), after exposure to elevated temperatures ranging from 200 to 800 degrees. These models were developed via adapting meta- heuristic models including the Water cycle algorithm (WCA), Genetic algorithm (GA), and classical AI models of Artificial neural networks (ANNs), Fuzzy logic models (FLM), in addition to the statistical method of Multiple linear regression (MLR). 156 post heating experimental results available as a literature data (represents four input parameters of temperature change, heat exposure duration, nanomaterial type, and replacement proportion) are used to achieve the study’s objective. Results of the developed models demonstrated that ANN and FLM have strong potential in predicting RCS. However, it is often infeasible to generate practical equations that relate input and output variables from these models. Upon analysing the results of the WCA and GA, it was found that WCA yielded the most accurate predictions based on all performance indicators. Furthermore, RCS prediction equations with superior accuracy were derived utilizing the meta-heuristic AI models of WCA and GA, with Mean absolute errors (MAEs) of 3.09 kg/cm² and 3.53 kg/cm² for the training, 1.91 kg/cm² and 2.72 kg/cm² for the validation, and 1.91 kg/cm² and 2.72 kg/cm² for the testing data sets, respectively. Additionally, sensitivity analysis via neural networks weights and SHAP investigation were performed to reveals the impact and relationship of the input variables with the output variables. Both techniques reveal that temperature degree and time of exposure had the highest positive impact on RCS value, followed by NAl and NCTs, in order.

A method for the dimensionality reduction of neural network weight matrices for natural language processing

The growth in the number of parameters of neural network models for language modeling makes the question of reducing the dimensionality of the matrices storing the models' weights increasingly relevant. The study proposes a novel method to reducing the dimensionality of training weight matrices and an algorithm called Convolutional Weights Compression. The essence of the method is to apply the convolution operation to the analysis of matrices of neural network training weights to reduce their dimensionality. В связи с ростом количества параметров нейросетевых моделей для обработки естественного языка все более актуальным становится вопрос о том, как уменьшить размерность матриц, в которых хранятся веса таких моделей. Исследование предлагает новый метод уменьшения размерности матриц обучающих весов и алгоритм сжатия под названием «сверточная компрессия весов». Суть метода заключается в применении операции свертки к анализу матриц обучающих весов нейронных сетей для уменьшения их размерности.

Finite-time optimal control for a class of nonlinear systems with performance constraints via critic-only ADP: Theory and experiments

This paper addresses the optimal control problem within the framework of adaptive dynamic programming (ADP) for a class of nonlinear systems subjected to performance constraints. A new finite-time optimal control scheme is developed to stabilize the system by using the critic-only neural network ADP method. Compared with the existing ADP-based optimal control methods with uniformly ultimately bounded stability, the provided control scheme ensures that the controlled system's state and neural network weight estimation error are finite-time stable. It can ensure optimality, prescribed performance, and finite-time stability of the closed-loop control system simultaneously through an integration of ADP, the prescribed performance control technique, and Lyapunov theory. The designed adaptive neural network weight update law can relax the persisting excitation condition. The proposed control scheme is implemented on a robotic experiment platform to achieve trajectory tracking and verify its effectiveness.

Adaptive impedance control method for manipulator based on radial basis function

Purpose In view of the unknown environmental parameters and uncertain interference during gripping by the manipulator, it is difficult to obtain an effective gripping force with the traditional impedance control method. To avoid this dilemma, the purpose of this study is to propose an adaptive control strategy based on an adaptive neural network and a PID search optimization algorithm for unknown environments. Design/methodology/approach The method is based on a variable impedance model, and a new impedance model is established using a radial basis function (RBF) neural network to estimate unknown parameters of the impedance model. The approximation errors of the adaptive neural network and the uncertain disturbance are effectively suppressed by designing the adaptive rate. In the meantime, auxiliary variables are constructed for Lyapunov stability analysis and adaptive controller design, and PSA is used to ensure the stability of the adaptive impedance control system. Based on the Lyapunov stability criterion, the adaptive im-pedance control system is proved to have progressive tracking convergence property. Findings Through comparative simulations and experiments, the superiority of the proposed adaptive control strategy in position and force tracking has been verified. For objects with low flexibility and light-weight (such as a coke, a banana and a nectarine), this control method demonstrates errors of less than 10%. Originality/value This paper uses RBF neural networks to estimate unknown parameters of the impedance model in real-time, enhancing system adaptability. Neural network weights are updated online to suppress errors and disturbances. Auxiliary variables are designed for Lyapunov stability analysis. The PSA algorithm is used to adjust controller parameters in real-time. Additionally, comparative simulations and experi-ments are designed to analyze and validate the performance of controller.

An efficient water quality index forecasting and categorization using optimized Deep Capsule Crystal Edge Graph neural network.

The world's freshwater supply, predominantly sourced from rivers, faces significant contamination from various economic activities, confirming that the quality of river water is critical for public health, environmental sustainability, and effective pollution control. This research addresses the urgent need for accurate and reliable water quality monitoring by introducing a novel method for estimating the water quality index (WQI). The proposed approach combines cutting-edge optimization techniques with Deep Capsule Crystal Edge Graph neural networks, marking a significant advancement in the field. The innovation lies in the integration of a Hybrid Crested Porcupine Genghis Khan Shark Optimization Algorithm for precise feature selection, ensuring that the most relevant indicators of water quality (WQ) are utilized. Furthermore, the use of the Greylag Goose Optimization Algorithm to fine-tune the neural network's weight parameters enhances the model's predictive accuracy. This dual optimization framework significantly improves WQI prediction, achieving a remarkable mean squared error (MSE) of 6.7 and an accuracy of 99%. By providing a robust and highly accurate method for WQ assessment, this research offers a powerful tool for environmental authorities to proactively manage river WQ, prevent pollution, and evaluate the success of restoration efforts. PRACTITIONER POINTS: Novel method combines optimization and Deep Capsule Crystal Edge Graph for WQI estimation. Preprocessing includes data cleanup and feature selection using advanced algorithms. Deep Capsule Crystal Edge Graph neural network predicts WQI with high accuracy. Greylag Goose Optimization fine-tunes network parameters for precise forecasts. Proposed method achieves low MSE of 6.7 and high accuracy of 99%.

Adaptive Inverse Nonlinear Optimal Control Based on Finite-Time Concurrent Learning and Semidefinite Programming.

This article investigates the problem of inverse optimal control (IOC) for a class of nonlinear affine systems. An adaptive IOC approach is proposed to recover the cost functional using only the system state data, which integrates the finite-time concurrent learning (FTCL) technique and the semidefinite programming (SDP) technique. First, an identifier neural network (NN) is employed to approximate the unknown nonlinear control policy, and an FTCL-based update law is proposed to estimate the weights of the identifier NN online, which removes the traditional persistent excitation (PE) condition. Moreover, the finite-time convergence as well as the uniformly ultimately boundness (UUB) of estimation error of the identifier NN weights are analysed according to whether or not there exists the identifier NN approximation error. Then, with the help of a value NN for approximating the value function, an SDP problem with a quadratic objective function can be set up for determining the weighting matrices of the cost functional. Finally, simulation results are presented to validate the proposed method.

E-commerce and Mobile Application Development in the Sports Industry

In order to achieve user recommendations that best match their current contextual needs, the author proposes a mobile service QoS (Quality of Service) hybrid recommendation model based on sports user situational awareness. Cluster the users and service items covered by mobile users based on their location contextual information according to the classification principle of autonomous systems, forming a collaborative filtering and recommendation mechanism for mobile user service items; In response to the cold start problem of new users and new projects in traditional QoS recommendation methods, prediction and recommendation of missing QoS attribute preference values are based on User based and Item based CF; In response to the problem of difficult determination of mixed recommendation weights caused by massive data and uneven distribution of service QoS attribute values in mobile network environments, MapReduce based ant colony neural network weight training is used for CF mixed recommendation. Experimental results have shown that the Hadoop sports industry has improved the operational efficiency of algorithms and reduced the time for global QoS preference prediction; And by comparing the operation results of the 10% and 100% sub datasets, it can also be seen that the acceleration ratio of the algorithm will continuously improve with the increase of data volume, thereby improving the operational efficiency of the recommendation algorithm. It has been proven that the MapReduce based ant colony neural network weight training method significantly reduces the global computation time of the algorithm and improves the operational efficiency of the recommendation system.

Recursive terminal sliding mode approximate optimal control strategy based on reconstructed nonlinear systems

For a class of nonlinear systems with unknown internal states and actuator failures, this paper proposes a recursive terminal sliding-mode fault-tolerant control method based on reconfigurable nonlinear systems and further designs an approximate optimal compensated controller based on adaptive dynamic programming, which employs a neural network (NN) to estimate the optimal value function. The system ensures that the tracking error converges to a small set around zero with faster speed and higher accuracy even when the actuator fails. Finally, the convergence of the expanded state observer and the single-evaluation NN weights is demonstrated based on the Lyapunov theory, and the stability of the whole closed-loop system is given. Simulation results and comparisons verify the effectiveness of the proposed control strategy.

Neural network optimal control for tripartite UAV confrontation systems based on fuzzy differential game

The neural network optimal control strategy based on a fuzzy differential game is proposed for the tripartite UAV confrontation systems consisting of the attackers, defenders, and targets. Firstly, the tripartite UAV mutual confrontation model is constructed and a nonlinear differential control system is established. Secondly, combining the fuzzy evaluation method and differential game theory, the tripartite UAV are divided into two parts of the confrontation game: attackers-defenders and attackers-targets. The optimal control strategies for the attackers, defenders and targets parties are derived separately. Then, the tripartite UAV game model is considered to be difficult to solve directly. The evaluation neural network is introduced to approximate the optimal value function using an adaptive dynamic programming method. The convergence of the evaluation neural network weights and the stability of the nonlinear differential control system are proved by using Lyapunov stability theory. Finally, the effectiveness of the tripartite UAV confrontation game control strategy designed in this paper is verified by simulation.

Improving model robustness to weight noise via consistency regularization

Abstract As an emerging computing architecture, the computing-in-memory (CIM) exhibits significant potential for energy efficiency and computing power in artificial intelligence applications. However, the intrinsic non-idealities of CIM devices, manifesting as random interference on the weights of neural network, may significantly impact the inference accuracy. In this paper, we propose a novel training algorithm designed to mitigate the impact of weight noise. The algorithm strategically minimizes cross-entropy loss while concurrently refining the feature representations in intermediate layers to emulate those of an ideal, noise-free network. This dual-objective approach not only preserves the accuracy of the neural network but also enhances its robustness against noise-induced degradation. Empirical validation across several benchmark datasets confirms that our algorithm sets a new benchmark for accuracy in CIM-enabled neural network applications. Compared to the most commonly used forward noise training methods, our approach yields approximately a 2% accuracy boost on the ResNet32 model with the CIFAR-10 dataset and a weight noise scale of 0.2, and achieves a minimum performance gain of 1% on ResNet18 with the ImageNet dataset under the same noise quantization conditions.

Robust neuro-adaptive command-filtered back-stepping fault-tolerant control of satellite using composite learning

This study proposes an adaptive neural command-filtered backstepping control system for precise and fast attitude control of a satellite considering different uncertain dynamics. Mission success crucially depends on robust attitude control resilient to model uncertainties, unmodeled dynamics, external disturbances, and actuator faults. The proposed approach synergistically combines a neural network for handling model uncertainties, unmodeled dynamics, and actuator faults, with a disturbance observer for compensating external disturbances and neural network estimation errors. The command-filtered backstepping technique avoids the explosion of complexity inherent to traditional backstepping, while integrating integral action into the design results in the elimination of the steady-state tracking error. Besides, a composite learning method optimizes the update laws for the neural network and disturbance observer weights, enhancing control performance. Despite the presence of uncertainties, the closed-loop system stability is guaranteed by the Lyapunov stability theorem. Simulation results demonstrate the proposed controller’s ability to handle severe actuator faults, unmodeled dynamics, and measurement noise without requiring explicit fault detection and isolation schemes.

Composite adaptive neural control for automatic carrier landing system with input saturation and output constraints

This paper investigates the automatic carrier landing control problem in the presence of model uncertainty, airwake disturbances, input saturation, and output constraints. Considering the performance requirements of the carrier-based aircraft, a composite adaptive neural controller is proposed based on the time-varying barrier Lyapunov function and backstepping control techniques. The radial basis function neural network is used to approximate the model uncertainty, where the neural network weight update law incorporating prediction and tracking errors further improves the convergence rate of the neural network and mitigates high-frequency oscillations. Furthermore, an adaptive disturbance compensation model is established to mitigate the adverse effects of airwake disturbances and estimation errors in the neural network. Based on the Lyapunov stability theory, it is proven that the proposed controller maintains the aircraft trajectory within the prescribed constraints and also ensures that all signals in the closed-loop control system are semiglobally uniformly ultimately bounded. Finally, comparative simulations are performed to demonstrate the effectiveness and superiority of the proposed composite adaptive neural control method.

Dependence of debris flow susceptibility maps on sampling strategy with data-driven grid-based model

Different sampling strategies produce varying sample data, serve as the primary input data and directly affect the accuracy of predictions in data-driven grid-based susceptibility models. This study analyzes the accuracy and variation of debris flow susceptibility maps (DFSMs) generated by various sampling strategies. The study area is the Yingxiu region in China, where six sampling strategies were applied, including three sampling locations (deposition area, runout area, and source area) and two sampling types (centroid and polygon) for the debris flow inventory. The effectiveness of 10 conditioning factors used to build the model was assessed by using Pearson correlation coefficient, variance inflation factor, and information gain ratio (IGR) techniques. We then used Weight of Evidence (WofE), Logistic Regression (LR), and Deep Neural Network (DNN) models to produce DFSMs and quantify their performance using the receiver operating characteristic curve (ROC), Accuracy (ACC), Precision, F1 score, and Recall. The results show that the WofE (AUC: 0.754–0.960), LR (AUC: 0.761–0.965), and DNN (AUC: 0.786–0.976) models all perform well, but the DFSMs and dominant factors depend strongly on sampling strategies, especially on sampling location. If the sample areas are excessively large and span across different factor class labels, or if there is a concentration of either large or small sample areas within a specific region, the results of centroid and polygon sampling strategies may differ or even be contradictory.We recommend: (1) determining sampling locations based on the research objectives to provide more accurate evaluation results; (2) selecting the sampling type by first considering the sample size. If the aforementioned conditions are not present, the quicker and more convenient centroid sampling strategy can be chosen; and (3) determining an appropriate sampling strategy and ensuring the accuracy of initial samples are paramount before producing DFSMs.

Neural learning control for sampled‐data nonlinear systems based on Euler approximation and first‐order filter

AbstractThe primary focus of this research paper is to explore the realm of dynamic learning in sampled‐data strict‐feedback nonlinear systems (SFNSs) by leveraging the capabilities of radial basis function (RBF) neural networks (NNs) under the framework of adaptive control. First, the exact discrete‐time model of the continuous‐time system is expressed as an Euler strict‐feedback model with a sampling approximation error. We provide the consistency condition that establishes the relationship between the exact model and the Euler model with meticulous detail. Meanwhile, a novel lemma is derived to show the stability condition of a digital first‐order filter. To address the non‐causality issues of SFNSs with sampling approximation error and the input data dimension explosion of NNs, the auxiliary digital first‐order filter and backstepping technology are combined to propose an adaptive neural dynamic surface control (ANDSC) scheme. Such a scheme avoids the ‐step time delays associated with the existing NN updating laws derived by the common ‐step predictor technology. A rigorous recursion method is employed to provide a comprehensive verification of the stability, guaranteeing its overall performance and dependability. Following that, the NN weight error systems are systematically decomposed into a sequence of linear time‐varying subsystems, allowing for a more detailed analysis and understanding. In order to ensure the recurrent nature of the input variables, a recursive design is employed, thereby satisfying the partial persistent excitation condition specifically designed for the RBF NNs. Meanwhile, it can verify that the NN estimated weights converge to their ideal values. Compared with the common ‐step predictor technology, there is no need to redesign the learning rules due to the designed NN weight updating laws without time delays. Subsequently, after capturing and storing the convergence weights, a novel neural learning dynamic surface control (NLDSC) scheme is specifically formulated by leveraging the acquired knowledge. The introduced methodology reduces computational complexity and facilitates practical implementation. Finally, empirical evidence obtained from simulation experiments validates the efficacy and viability of the proposed methodology.

Adaptive Finite-Time-Based Neural Optimal Control of Time-Delayed Wheeled Mobile Robotics Systems.

For nonlinear systems with uncertain state time delays, an adaptive neural optimal tracking control method based on finite time is designed. With the help of the appropriate LKFs, the time-delay problem is handled. A novel nonquadratic Hamilton-Jacobi-Bellman (HJB) function is defined, where finite time is selected as the upper limit of integration. This function contains information on the state time delay, while also maintaining the basic information. To meet specific requirements, the integral reinforcement learning method is employed to solve the ideal HJB function. Then, a tracking controller is designed to ensure finite-time convergence and optimization of the controlled system. This involves the evaluation and execution of gradient descent updates of neural network weights based on a reinforcement learning architecture. The semi-global practical finite-time stability of the controlled system and the finite-time convergence of the tracking error are guaranteed.

A combined model based on decomposition and reorganization, weight optimization algorithms for carbon price point and interval prediction

With the increasing environmental and climate problems caused by global warming, more and more countries are paying attention to reducing carbon emissions. Accurately predicting carbon prices can help create a stable carbon market and mitigate global warming. Therefore, a combined model which has four modules is proposed, they are the decomposition of carbon price series, reorganization of subsequences, point forecasting, and interval forecasting. For data preprocessing of decomposition, it can reduce the non-linearity of sequences. Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN) decomposes the original series into multiple Intrinsic Mode Functions (IMFs). Reorganization uses SampEn (SE) and T-test to reorganize the decomposed sequence into three new sequences according to actual meanings. For point prediction, the combined prediction model uses Particle Swarm Optimization (PSO) to obtain the optimal weights of Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU), which can integrate their strengths to improve prediction accuracy. Interval prediction is based on fitting the optimal distribution function by Maximum Likelihood Estimate (MLE). Different significance intervals can be obtained from the point prediction results and the distribution function. Empirical and comparative experiments show this combined model has excellent point and interval predictions. Besides, the significance and stability are demonstrated. Finally, some valuable references are provided for policymakers.

Optimal Asymptotic Tracking Control for Nonzero-Sum Differential Game Systems with Unknown Drift Dynamics via Integral Reinforcement Learning

This paper employs an integral reinforcement learning (IRL) method to investigate the optimal tracking control problem (OTCP) for nonlinear nonzero-sum (NZS) differential game systems with unknown drift dynamics. Unlike existing methods, which can only bound the tracking error, the proposed approach ensures that the tracking error asymptotically converges to zero. This study begins by constructing an augmented system using the tracking error and reference signal, transforming the original OTCP into solving the coupled Hamilton–Jacobi (HJ) equation of the augmented system. Because the HJ equation contains unknown drift dynamics and cannot be directly solved, the IRL method is utilized to convert the HJ equation into an equivalent equation without unknown drift dynamics. To solve this equation, a critic neural network (NN) is employed to approximate the complex value function based on the tracking error and reference information data. For the unknown NN weights, the least squares (LS) method is used to design an estimation law, and the convergence of the weight estimation error is subsequently proven. The approximate solution of optimal control converges to the Nash equilibrium, and the tracking error asymptotically converges to zero in the closed system. Finally, we validate the effectiveness of the proposed method in this paper based on MATLAB using the ode45 method and least squares method to execute Algorithm 2.

Deep neural network homogenization of multiphysics behavior for periodic piezoelectric composites

We present a physically informed deep neural network homogenization theory for identifying homogenized moduli and local electromechanical fields of periodic piezoelectric composites. The theory employs a multi-network model to represent solutions to stress equilibrium and electrostatic partial differential equations in the inclusion and the matrix phases, leading to a more accurate depiction of field variables. Satisfaction of periodicity conditions for hexagonal and cubic symmetries are efficiently tackled using a series of sinusoidal functions with known periods and adjustable parameters. Additionally, the theory applies fully trainable weights to the collocation points. These trainable weights, trained concurrently with the neural network weights, compel the neural networks to enhance their performance when faced with large local stress/deformation gradients. The predictive capability of the proposed theory is illustrated by comparison with finite-element solutions for composites reinforced with unidirectional fiber, spherical particle, or weakened by an ellipsoidal cavity over a wide range of volume fractions.

Neural networks-based composite learning control for robotic systems with predefined time error constraints

This paper investigates the neural networks-based control problem for robotic systems with error constraints. By embedding a new predefined time performance function into an asymmetric barrier function, a predefined time constraints method is proposed with the following features: (1) it is a general approach that can handle asymmetric error constraints and can be directly applied to the predefined performance control for other high-order nonlinear systems without any changing; (2) both settling time and convergent compact set can be preassigned by the user; (3) the results are global i.e., the robot can start arbitrarily within the physical domain. By extending the regression operators with online data memory, a prediction error is constructed, by using which, a new neural networks based composite learning law is constructed to handle unknown dynamics. Compared with the traditional method such as gradient descent algorithm, σ-modification and ϵ-modification which cannot guarantee the convergence of the neural networks weights or can only force the weights to stay around preselected values, the neural networks guarantees that the weights converge to the region around the true values, in particular under a weak interval excitation (IE) condition rather than the typically stringent persistent excitation (PE) condition. The benefits and effectiveness of the proposed control are theoretically authenticated and experimentally validated.

ATHENA-FL: Avoiding Statistical Heterogeneity with One-versus-All in Federated Learning

Federated learning (FL) is a distributed approach to train machine learning models without disclosing private data from participating clients to a central server. Nevertheless, FL training struggles to converge when clients have distinct data distributions, which leads to an increased training time and model prediction error. We propose ATHENA-FL, a federated learning system that considers clients with heterogeneous data distributions to generate accurate models in fewer training epochs than state-of-the-art approaches. ATHENA-FL reduces communication costs, providing an additional positive aspect for resource-constrained scenarios. ATHENA-FL mitigates data heterogeneity by introducing a preliminary step before training that clusters clients with similar data distribution. To handle that, we use the weights of a locally trained neural network used as a probe. The proposed system also uses the one-versus-all model to train one binary detector for each class in the cluster. Thus, clients can compose complex models combining multiple detectors. These detectors are shared with all participants through the system's database. We evaluate the clustering procedure using different layers from the neural network and verify that the last layer is sufficient to cluster the clients efficiently. The experiments show that using the last layer as input for the clustering algorithm transmits 99.68% fewer bytes to generate clusters compared to using all the neural network weights. Finally, our results show that ATHENA-FL correctly identifies samples, achieving up to 10.9% higher accuracy than traditional training. Furthermore, ATHENA-FL achieves lower training communication costs compared with MobileNet architecture, reducing the number of transmitted bytes between 25% and 97% across evaluated scenarios.