Network Weights Research Articles

DEWNA: dynamic entropy weight network analysis and its application to the DNA-binding proteome in A549 cells with cisplatin-induced damage.

Cisplatin is one of the most commonly used chemotherapy drugs for treating solid tumors. As a genotoxic agent, cisplatin binds to DNA and forms platinum-DNA adducts that cause DNA damage and activate a series of signaling pathways mediated by various DNA-binding proteins (DBPs), ultimately leading to cell death. Therefore, DBPs play crucial roles in the cellular response to cisplatin and in determining cell fate. However, systematic studies of DBPs responding to cisplatin damage and their temporal dynamics are still lacking. To address this, we developed a novel and user-friendly stand-alone software, DEWNA, designed for dynamic entropy weight network analysis to reveal the dynamic changes of DBPs and their functions. DEWNA utilizes the entropy weight method, multiscale embedded gene co-expression network analysis and generalized reporter score-based analysis to process time-course proteome expression data, helping scientists identify protein hubs and pathway entropy profiles during disease progression. We applied DEWNA to a dataset of DBPs from A549 cells responding to cisplatin-induced damage across 8 time points, with data generated by data-independent acquisition mass spectrometry (DIA-MS). The results demonstrate that DEWNA can effectively identify protein hubs and associated pathways that are significantly altered in response to cisplatin-induced DNA damage, and offer a comprehensive view of how different pathways interact and respond dynamically over time to cisplatin treatment. Notably, we observed the dynamic activation of distinct DNA repair pathways and cell death mechanisms during the drug treatment time course, providing new insights into the molecular mechanisms underlying the cellular response to DNA damage.

Novel adaptive MPPT technique for enhanced performance of grid integrated solar photovoltaic system

In order to extract the maximum photovoltaic (PV) power under stochastic weather conditions, a maximum power point tracking (MPPT) technique is required. PV modules behave nonlinearly in unpredictable weather conditions, and the effectiveness of many control strategies under such conditions especially conventional ones—decreases significantly. It is essential to maximize the use of PV power in the system; thus, MPPT algorithms have been developed to ensure that grid integrated PV systems work best at the maximum power point regardless of weather conditions. This research presents a novel HLO-ANN (Horned Lizard optimized artificial neural network) MPPT technique for grid-integrated solar photovoltaic systems which is the improved version of the ANN-MPPT, developed using a novel HLO technique, which effectively optimizes weights of the neural network. The irradiance and temperature datasets from the NASA website have been utilized to train the neural network. The proposed methods have been evaluated by comparing their simulation results to other ANN optimized MPPT techniques: The results showed that the proposed method outperforms the other methods in terms of robustness, DC bus voltage regulation, tracking speed, convergence to the minimum error value, and tracking efficiency in maximizing the harvested power from the PV systems. Furthermore, the EN50530 MPPT efficiency test was carried out during both fast and slow-varying irradiance levels to examine the effectiveness of proposed algorithms.

Ship target detection in SAR images based on SimAM attention YOLOv8

AbstractDeep learning has been widely applied in ship detection in synthetic aperture radar (SAR) imagery due to their powerful feature representation capabilities. However, YOLOv8 models treat all regions of the image equally during convolutional feature processing, resulting in less‐than‐ideal outcomes. To address this limitation, this study proposes a simple, parameter‐free attention module (SimAM) attention‐based YOLOv8 algorithm for ship detection in SAR images. The proposed algorithm first passes through a backbone network, which incorporates SimAM attention modules. The SimAM attention mechanism successfully allocates the convolutional neural network's 3D weights effectively using an energy function method, without introducing additional parameters. This mechanism enables the network to automatically emphasize key features in the image, enhancing its ability to represent target areas and suppress background interference. Subsequently, deep features are upsampled and fused with relatively shallow features to extract features at three different scales and achieve target detection, ultimately outputting classification and positional information of the targets. The effectiveness of the model on the SAR‐ship‐dataset is experimentally validated achieving an mAP50 value of 97.72% and an mAP50‐95 value of 68.99%, confirming the superiority of the proposed model.

Adaptive dynamic programming–based fault-tolerant control of multi-UAV formation system

With the increasing utilization of unmanned aerial vehicle (UAV) swarms in civilian and military field applications, it is crucial to eliminate the potential impacts of UAV faults on task completion and efficiency. To address this issue, the present paper investigates a fault-tolerant control scheme based on the adaptive dynamic programming (ADP) method and fault observer. The whole closed-loop system is composed of a position loop and an attitude loop. An ADP-based position-tracking controller is established with an improved cost function that can simultaneously represent actuator faults, control inputs, and tracking errors. In addition, a single-layer critic neural network is constructed to estimate the cost function and approximate control inputs. In the process of adjusting the weights of the neural network, a unique adjustment method combining policy gradient and prioritized experience replay is adopted to improve the data usage efficiency and speed up the weight convergence. Furthermore, a sliding mode–based attitude controller is utilized for the inner-loop attitude subsystem. The significant feature of this scheme is that it has the ability to self-learn and self-adapt. A Lyapunov stability analysis is performed to verify whether the signals are uniformly ultimately bounded. Finally, simulation experiments are conducted to demonstrate the effectiveness of the proposed scheme.

A discrete element solution method embedded within a Neural Network

This paper introduces a novel methodology, the Neural Network framework for the Discrete Element Method (NN4DEM), as part of a broader initiative to harness specialised AI hardware and software environments, marking a transition from traditional computational physics programming approaches. NN4DEM enables GPU-parallelised computations by mapping particle data (coordinates and velocities) onto uniform grids as solution fields and computing contact forces by applying mathematical operations that can be found in convolutional neural networks (CNN). Essentially, this framework transforms a DEM problem into a series of layered “images” composed of pixels, using stencil operations to compute the DEM physics, which is inherently local. The method revolves around custom kernels, with operations prescribed by the laws of physics for contact detection and interaction. Therefore, unlike conventional AI methods, it eliminates the need for training data to determine network weights. NN4DEM utilises libraries such as PyTorch for relatively easier programmability and platform interoperability. This paper presents the theoretical foundations, implementation and validation of NN4DEM through hopper test benchmarks. An analysis of the results from random packing cases highlights the ability of NN4DEM to scale to 3D models with millions of particles. The paper concludes with potential research directions, including further integration with other physics-based models and applications across various multidisciplinary fields.

A crack identification scheme based on neural network surrogate model and XFEM

Crack detection and identification is of great significance to the safety issues of engineering structures. In this paper, an intelligent crack identification scheme based on extended finite element and neural network surrogate model is proposed to realize the accurate identification of crack parameters. The method firstly employs extended finite element forward analysis to obtain the displacement data of measurement points on geometric models with different crack lengths, and inputs them as sample data to train the agent model, establishes a neural network-based inverse analysis model for crack identification, and automatically updates the threshold and weight of the neural network by using the Gray Wolf optimization algorithm to finally compute the globally optimal results. In the screening of the surrogate model, this paper verifies the advantages of the neural network surrogate model in data fitting and crack information extraction by comparing and analyzing the characteristics of neural network, support vector machine and other surrogate models, and optimizing the neural network surrogate model by adopting the Gray Wolf optimization algorithm. Finally, several numerical examples of different types of cracks are given to verify the validity of the proposed method, and the results show that the proposed method can accurately invert the geometric information of cracks.

Neural network optimal control for tripartite UAV confrontation systems based on fuzzy differential game

The neural network optimal control strategy based on a fuzzy differential game is proposed for the tripartite UAV confrontation systems consisting of the attackers, defenders, and targets. Firstly, the tripartite UAV mutual confrontation model is constructed and a nonlinear differential control system is established. Secondly, combining the fuzzy evaluation method and differential game theory, the tripartite UAV are divided into two parts of the confrontation game: attackers-defenders and attackers-targets. The optimal control strategies for the attackers, defenders and targets parties are derived separately. Then, the tripartite UAV game model is considered to be difficult to solve directly. The evaluation neural network is introduced to approximate the optimal value function using an adaptive dynamic programming method. The convergence of the evaluation neural network weights and the stability of the nonlinear differential control system are proved by using Lyapunov stability theory. Finally, the effectiveness of the tripartite UAV confrontation game control strategy designed in this paper is verified by simulation.

Control of thermal uniformity in microwave heating process by BPNN and adaptive particle swarm optimization

As China advances its green economy, innovative methods are being employed to enhance energy optimization and conservation within energy-intensive industries. Among these methods, microwave heating stands out due to its superior heating efficiency and lack of pollution. Nonetheless, uniform heating remains a challenge because the microwave absorption capacity of the heated medium varies with changes in heating time and temperature. To address this issue, an Adaptive Particle Swarm Optimization (APSO) neural network microwave system based on the Back Propagation Neural Network (BPNN) is proposed. This system leverages the fundamental principles of Particle Swarm Optimization (PSO), categorizing particle swarms into three types and iterating them through distinct processes to achieve optimal results. An APSO controller is designed based on system identification, adjusting the controller parameters according to the error between the real-time system output and the identification model. The feedback error is used as the fitness function in the PSO algorithm, continuously adjusting the weights and thresholds of the neural network. This intelligent control approach optimizes the microwave oven's input power to minimize the error between the actual temperature output and the identified temperature. The APSO controller is designed based on system identification, with the intelligently controlled microwave heating system adjusting its input power to minimize the error between the identified and actual temperature outputs. Unlike traditional Proportional Integral Differential (PID) and BPNN controllers, this approach calculates the output of the identified model and the error of the actual model, feeding this information back to the controller. The feedback error serves as the fitness function in the PSO algorithm, enabling continuous adjustment of the network's weights and thresholds to regulate the microwave equipment's output power, thereby ensuring the output temperature closely follows the preset temperature curve. Experimental results demonstrate that the actual output value of the system closely aligns with the original preset temperature curve, with a root mean square error of only 0.74. This designed identification and control system effectively achieves the desired outcome.

Obtaining and qualitative analysis of time-lagged correlations between seawater quality parameters

Abstract In the regulation of seawater quality, it is crucial to understand the interactions between parameters and the time-lagged effects. This paper focuses on the problem of how to obtain and analyze time-lagged correlations between seawater quality parameters, an issue that has not attracted the attention of researchers. In this paper, a graph neural network-based model, Dynamic Adjacency Weight Network, is proposed to model the time-lagged correlation between seawater quality parameters. By regarding the parameters as nodes, the proposed model retains the relationships between the parameters in a weighted adjacency matrix, overcoming the problem of traditional deep neural networks that are difficult to be visualized. Meanwhile, the proposed multivariate multistep shift prediction strategy enables the proposed model to accurately obtain the time-lagged correlation information between parameters at different time intervals. In addition, the proposed model also addresses some of the details worth considering when obtaining correlations between seawater quality parameters. The proposed model performs well in the prediction of water quality parameters. This indirectly verifies the validity of the obtained correlations and overcomes the difficulty of verifying the validity of statistical methods. This study provides new ideas and methods for seawater quality monitoring and research.

Genotype and Phenotype Association Analysis Based on Multi-omics Statistical Data

Background: When using clinical data for multi-omics analysis, there are issues such as the insufficient number of omics data types and relatively small sample size due to the protection of patients' privacy, the requirements of data management by various institutions, and the relatively large number of features of each omics data. This paper describes the analysis of multi-omics pathway relationships using statistical data in the absence of clinical data. Methods: We proposed a novel approach to exploit easily accessible statistics in public databases. This approach introduces phenotypic associations that are not included in the clinical data and uses these data to build a three-layer heterogeneous network. To simplify the analysis, we decomposed the three-layer network into double two-layer networks to predict the weights of the inter-layer associations. By adding a hyperparameter β, the weights of the two layers of the network were merged, and then k-fold cross-validation was used to evaluate the accuracy of this method. In calculating the weights of the two-layer networks, the RWR with fixed restart probability was combined with PBMDA and CIPHER to generate the PCRWR with biased weights and improved accuracy. Results: The area under the receiver operating characteristic curve was increased by approximately 7% in the case of the RWR with initial weights. Conclusion: Multi-omics statistical data were used to establish genotype and phenotype correlation networks for analysis, which was similar to the effect of clinical multi-omics analysis.

Improving model robustness to weight noise via consistency regularization

Abstract As an emerging computing architecture, the computing-in-memory (CIM) exhibits significant potential for energy efficiency and computing power in artificial intelligence applications. However, the intrinsic non-idealities of CIM devices, manifesting as random interference on the weights of neural network, may significantly impact the inference accuracy. In this paper, we propose a novel training algorithm designed to mitigate the impact of weight noise. The algorithm strategically minimizes cross-entropy loss while concurrently refining the feature representations in intermediate layers to emulate those of an ideal, noise-free network. This dual-objective approach not only preserves the accuracy of the neural network but also enhances its robustness against noise-induced degradation. Empirical validation across several benchmark datasets confirms that our algorithm sets a new benchmark for accuracy in CIM-enabled neural network applications. Compared to the most commonly used forward noise training methods, our approach yields approximately a 2% accuracy boost on the ResNet32 model with the CIFAR-10 dataset and a weight noise scale of 0.2, and achieves a minimum performance gain of 1% on ResNet18 with the ImageNet dataset under the same noise quantization conditions.

ADP-based online compensation hierarchical sliding-mode control for partially unknown switched nonlinear systems with actuator failures

This article investigates an adaptive dynamic programming-based online compensation hierarchical sliding-mode control problem for a class of partially unknown switched nonlinear systems with actuator failures and uncertain perturbations under an identifier-critic neural networks architecture. Firstly, by introducing a cost function related to hierarchical sliding-mode surfaces for the nominal system, the original control problem is equivalently converted into an optimal control problem. To obtain this optimal control policy, the Hamilton–Jacobi–Bellman equation is solved through an adaptive dynamic programming method. Compared with conventional adaptive dynamic programming methods, the identifier-critic network architecture not only overcomes the limitation on the unknown internal dynamic but also eliminates the approximation error arising from the actor network. The weights in the critic network are tuned via the gradient descent approach and the experience replay technology, such that the persistence of excitation condition can be relaxed. Then, a compensation term containing hierarchical sliding-mode surfaces is used to offset uncertain actuator failures without the fault detection and isolation unit. Based on the Lyapunov stability theory, all states of the closed-loop nonlinear system are stable in the sense of uniformly ultimately boundedness. Finally, numerical and practical examples are given to demonstrate the effectiveness of our presented online compensation control strategy.

A novel load allocation strategy based on the adaptive chiller model with data augmentation

Model-based load allocation strategy is an impactful solution to enhance energy efficiency of multiple-chiller system. Its performance is heavily dependent on the accuracy of chiller model. Data-driven model is a pretty-good solution. However, in real multiple-chiller system, the range of operation condition in historical data is commonly narrow, so it is challenging to develop an accurate data-driven model of chiller throughout full range of operation condition. In this paper, data augmentation algorithm is presented to generate the data outside of historical data, which is based on conditional generative adversarial network (CGAN) and elastic weight consolidation algorithm (EWC). Combined historical data and generated data, augmented training dataset is set up and updated by online operation data. Trained by online updated augmented training dataset periodically, adaptive chiller model is set up. Based on adaptive chiller model, a novel load allocation strategy presented for multiple-chiller system. The proposed strategy is verified by field test in multiple-chiller system. The results show that adaptive chiller model, with the aid of data augmentation algorithm, is more accurate. The proposed strategy can achieve 5.03 % energy saving compared with fixed set-point strategy, and the EER of proposed strategy is 6.27 % higher than that of fixed set-point strategy.

Robust neuro-adaptive command-filtered back-stepping fault-tolerant control of satellite using composite learning

This study proposes an adaptive neural command-filtered backstepping control system for precise and fast attitude control of a satellite considering different uncertain dynamics. Mission success crucially depends on robust attitude control resilient to model uncertainties, unmodeled dynamics, external disturbances, and actuator faults. The proposed approach synergistically combines a neural network for handling model uncertainties, unmodeled dynamics, and actuator faults, with a disturbance observer for compensating external disturbances and neural network estimation errors. The command-filtered backstepping technique avoids the explosion of complexity inherent to traditional backstepping, while integrating integral action into the design results in the elimination of the steady-state tracking error. Besides, a composite learning method optimizes the update laws for the neural network and disturbance observer weights, enhancing control performance. Despite the presence of uncertainties, the closed-loop system stability is guaranteed by the Lyapunov stability theorem. Simulation results demonstrate the proposed controller’s ability to handle severe actuator faults, unmodeled dynamics, and measurement noise without requiring explicit fault detection and isolation schemes.

Fast Yet Effective Machine Unlearning.

Unlearning the data observed during the training of a machine learning (ML) model is an important task that can play a pivotal role in fortifying the privacy and security of ML-based applications. This article raises the following questions: 1) can we unlearn a single or multiple class(es) of data from an ML model without looking at the full training data even once? and 2) can we make the process of unlearning fast and scalable to large datasets, and generalize it to different deep networks? We introduce a novel machine unlearning framework with error-maximizing noise generation and impair-repair based weight manipulation that offers an efficient solution to the above questions. An error-maximizing noise matrix is learned for the class to be unlearned using the original model. The noise matrix is used to manipulate the model weights to unlearn the targeted class of data. We introduce impair and repair steps for a controlled manipulation of the network weights. In the impair step, the noise matrix along with a very high learning rate is used to induce sharp unlearning in the model. Thereafter, the repair step is used to regain the overall performance. With very few update steps, we show excellent unlearning while substantially retaining the overall model accuracy. Unlearning multiple classes requires a similar number of update steps as for a single class, making our approach scalable to large problems. Our method is quite efficient in comparison to the existing methods, works for multiclass unlearning, does not put any constraints on the original optimization mechanism or network design, and works well in both small and large-scale vision tasks. This work is an important step toward fast and easy implementation of unlearning in deep networks. Source code: https://github.com/vikram2000b/Fast-Machine-Unlearning.

Towards Understanding Convergence and Generalization of AdamW.

AdamW modifies Adam by adding a decoupled weight decay to decay network weights per training iteration. For adaptive algorithms, this decoupled weight decay does not affect specific optimization steps, and differs from the widely used l2-regularizer which changes optimization steps via changing the first- and second-order gradient moments. Despite its great practical success, for AdamW, its convergence behavior and generalization improvement over Adam and l2-regularized Adam ( l2-Adam) remain absent yet. To solve this issue, we prove the convergence of AdamW and justify its generalization advantages over Adam and l2-Adam. Specifically, AdamW provably converges but minimizes a dynamically regularized loss that combines vanilla loss and a dynamical regularization induced by decoupled weight decay, thus yielding different behaviors with Adam and l2-Adam. Moreover, on both general nonconvex problems and PŁ-conditioned problems, we establish stochastic gradient complexity of AdamW to find a stationary point. Such complexity is also applicable to Adam and l2-Adam, and improves their previously known complexity, especially for over-parametrized networks. Besides, we prove that AdamW enjoys smaller generalization errors than Adam and l2-Adam from the Bayesian posterior aspect. This result, for the first time, explicitly reveals the benefits of decoupled weight decay in AdamW. Experimental results validate our theory.

Wind power forecasting based on ensemble deep learning with surrogate-assisted evolutionary neural architecture search and many-objective federated learning

Deep neural networks (DNN) have been widely used in wind power forecasting (WPF), however, they still encounter difficulties such as delayed variable selection, DNN architecture design, and network weight optimization. Therefore, an ensemble cascaded DNN (ECDNN) with surrogate-assisted evolutionary neural architecture search (SEANS) and many-objective federated learning (MOFL) is proposed and referred to as ECDNN-SENAS-MOFL. SEANS is proposed to determine the delayed variables and network architecture efficiently, which allows significantly enhancing the evolutionary search efficiency than traditional evolutionary neural architecture search (ENAS) approaches by employing a surrogate model to achieve fast fitness evaluations. Meanwhile, MOFL is developed by embedding the federated learning into many-objective optimization to obtain more reliable network weights based on the distributed modeling data than traditional centralized learning and federated learning, as well as solving the problems of data privacy protection and data islands. Furthermore, selective ensemble learning is introduced to build high-performance ensemble WPF model by combining the merits of multiple highly influential Pareto solutions. The effectiveness and superiority of the proposed ECDNN-SENAS-MOFL method are verified using an actual wind power dataset. The application results show that the proposed method can provide new insights into data-driven WPF modeling and has a great potential for practical applications.

Composite adaptive neural control for automatic carrier landing system with input saturation and output constraints

This paper investigates the automatic carrier landing control problem in the presence of model uncertainty, airwake disturbances, input saturation, and output constraints. Considering the performance requirements of the carrier-based aircraft, a composite adaptive neural controller is proposed based on the time-varying barrier Lyapunov function and backstepping control techniques. The radial basis function neural network is used to approximate the model uncertainty, where the neural network weight update law incorporating prediction and tracking errors further improves the convergence rate of the neural network and mitigates high-frequency oscillations. Furthermore, an adaptive disturbance compensation model is established to mitigate the adverse effects of airwake disturbances and estimation errors in the neural network. Based on the Lyapunov stability theory, it is proven that the proposed controller maintains the aircraft trajectory within the prescribed constraints and also ensures that all signals in the closed-loop control system are semiglobally uniformly ultimately bounded. Finally, comparative simulations are performed to demonstrate the effectiveness and superiority of the proposed composite adaptive neural control method.

Adaptive structure generation and neuronal differentiation for memory encoding in SNNs

Memory is the core of cognition. The exploration of the memory encoding mechanism or the representation mechanism of information in the Spiking Neural Network (SNN) is the basis for the in-depth study of memory. In this paper, we study the memory encoding mechanism of multilayer SNN models from a biomimetic perspective and explore a method using the high biological likelihood of SNN to enable the network to effectively simulate memory effects. We proposed a series of heuristic neuron-growing connection algorithms and supervised network weight learning algorithms, which were applied to the unsupervised and supervised training process of the presentation layer. These methods optimized the structure of the representation layer, achieved functional differentiation of neurons, and enabled the network to generate differentiated representations for different data modes. Under our algorithm, the proposed model achieves stable convergence with identical pattern inputs, demonstrating distinct representations and sensitivities to different visual modalities. To achieve stable information expression within the network, we conducted various comparative experiments to determine diverse parameters of the complex network. This paper contributes to the development of Brain-inspired Intelligence by bridging the gap between computer science and neuroscience by using simulations to validate biological hypotheses and guide machine learning.

Dependence of debris flow susceptibility maps on sampling strategy with data-driven grid-based model

Different sampling strategies produce varying sample data, serve as the primary input data and directly affect the accuracy of predictions in data-driven grid-based susceptibility models. This study analyzes the accuracy and variation of debris flow susceptibility maps (DFSMs) generated by various sampling strategies. The study area is the Yingxiu region in China, where six sampling strategies were applied, including three sampling locations (deposition area, runout area, and source area) and two sampling types (centroid and polygon) for the debris flow inventory. The effectiveness of 10 conditioning factors used to build the model was assessed by using Pearson correlation coefficient, variance inflation factor, and information gain ratio (IGR) techniques. We then used Weight of Evidence (WofE), Logistic Regression (LR), and Deep Neural Network (DNN) models to produce DFSMs and quantify their performance using the receiver operating characteristic curve (ROC), Accuracy (ACC), Precision, F1 score, and Recall. The results show that the WofE (AUC: 0.754–0.960), LR (AUC: 0.761–0.965), and DNN (AUC: 0.786–0.976) models all perform well, but the DFSMs and dominant factors depend strongly on sampling strategies, especially on sampling location. If the sample areas are excessively large and span across different factor class labels, or if there is a concentration of either large or small sample areas within a specific region, the results of centroid and polygon sampling strategies may differ or even be contradictory.We recommend: (1) determining sampling locations based on the research objectives to provide more accurate evaluation results; (2) selecting the sampling type by first considering the sample size. If the aforementioned conditions are not present, the quicker and more convenient centroid sampling strategy can be chosen; and (3) determining an appropriate sampling strategy and ensuring the accuracy of initial samples are paramount before producing DFSMs.