Parameter Update Research Articles

A sequential multi-agent reinforcement learning framework for different action spaces

In many real-world decision-making tasks, multi-agent need to learn collaboration in a high-dimensional complex action space, rather than just a single discrete action space. Recently, value decomposition learning methods such as QMIX have emerged as a promising approaches for collaborative multi-agent tasks. However, most of the value decomposition algorithms can only be used in discrete action space, which would limit their practicability. To address the limitation, we propose a novel algorithm called Multi-Agent Sequential Q-Networks (MASQN), which can be applied to the multi-agent domains with continuous, multidiscrete or hybrid action spaces. The proposed algorithm is based on the structure of centralized training with decentralized execution (CTDE). The decentralized actors ensure adaptability to different action spaces by utilizing action space discretization and sequential models, and the centralized critic utilizes the value decomposition architecture to guide effective updates of the policy parameters for each agent. We also give the convergence of joint policy from the perspective of policy iteration, by combining it with the CTDE structure and the constraint of the Individual Global Max (IGM) condition. Finally, we evaluate the MASQN algorithm on two benchmark environments: MAMuJoCo and Hybrid Predator–Prey. The empirical results show that MASQN out performs the state-of-the-art performance on three different action spaces.

Mutual information-based feature selection for inverse mapping parameter updating of dynamical systems

AbstractA digital twin should be and remain an accurate model representation of a physical system throughout its operational life. To this end, we aim to update (physically interpretable) parameters of such a model in an online fashion. Hereto, we employ the inverse mapping parameter updating (IMPU) method that uses an artificial neural network (ANN) to map features, extracted from measurement data, to parameter estimates. This is achieved by training the ANN offline on simulated data, i.e., pairs of known parameter value sets and sets of features extracted from corresponding simulations. Since a plethora of features (and feature types) can be extracted from simulated time domain data, feature selection (FS) strategies are investigated. These strategies employ the mutual information between features and parameters to select an informative subset of features. Hereby, accuracy of the parameters estimated by the ANN is increased and, at the same time, ANN training and inference computation times are decreased. Additionally, Bayesian search-based hyperparameter tuning is employed to enhance performance of the ANNs and to optimize the ANN structure for various FS strategies. Finally, the IMPU method is applied to a high-tech industrial use case of a semi-conductor machine, for which measurements are performed in closed-loop on the controlled physical system. This system is modeled as a nonlinear multibody model in the Simscape multibody environment. It is shown that the model updated using the IMPU method simulates the measured system more accurately than a reference model of which the parameter values have been determined manually.

Research on Vehicle Stability Control Based on a Union Disturbance Observer and Improved Adaptive Unscented Kalman Filter

This paper considers external disturbances imposed on vehicle systems. Based on a vehicle dynamics model of the vehicle with three degrees of freedom (3-DOFs), a union disturbance observer (UDO) composed of a nonlinear disturbance observer (NDO) and an extended state observer (ESO) was designed to obtain external disturbances and unmodeled items. Meanwhile, an improved adaptive unscented Kalman filter (iAUKF) with anti-disturbance and anti-noise properties is proposed, based on the UDO and the unscented Kalman filter (UKF) method, to evaluate the sideslip angle of vehicle systems. Finally, a vehicle yaw stability controller was designed based on UDO and the global fast terminal sliding mode control (GFTSMC) method. The results of co-simulation demonstrated that the proposed UDO was effectively able to observe external disturbances and unmodeled items. The proposed iAUKF, which considers external disturbances, not only achieves adaptive updating and adjustment of filtering parameters under different sensor noise intensities but can also resist external disturbances, improving the estimation accuracy and robustness of the UKF. In the anti-disturbance performance test, the maximum estimation error of the sideslip angle of the iAUKF under the three working conditions was less than 0.1°, 0.02°, and 0.5°, respectively. Based on the UDO and the GFTSMC, a vehicle yaw stability controller is described, which improves the accuracy of control and the robustness of the vehicle’s stability control system and greatly strengthens the driving safety of the vehicle.

Comparison of barrier update strategies for interior point algorithms in single-crystal plasticity

This contribution discusses the influence of different barrier update strategies on the performance and robustness of an interior point algorithm for single-crystal plasticity at small strains. To this end, single-crystal plasticity is first briefly presented in the framework of a primal-dual interior point algorithm to outline the general algorithmic structure. The manner in which the barrier parameter is modified within the interior point method, steering the penalization of constraints, plays a crucial role for the robustness and efficiency of the overall algorithm. In this paper, we compare and analyze different strategies in the framework of crystal plasticity. In a thorough analysis of a numerical example covering a broad range of settings in monocrystals, we investigate robust hyperparameter ranges and identify the most efficient and robust barrier parameter update strategies.

Composite learning prescribed performance control of strict-feedback nonlinear systems with mismatched parametric uncertainties

A composite learning prescribed performance control (CLPPC) approach is presented for strict-feedback nonlinear systems with mismatched parametric uncertainties. A finite-time performance function based on polynomial is introduced to predefined a restriction region with respect to the tracking error. By introducing an error transformation function, the tracking error restriction problem of the original system is transformed into the stability problem of an equivalent transformation system. To guarantee the convergence of unknown parameters, online recording data and instantaneous are used to generate a prediction error, which is used together with filtered tracking errors to update parameter estimations under an interval excitation condition. All signals are proven to be ultimately uniformly stable under the proposed CLPPC strategy. Furthermore, the tracking error is limited within the predefined region after the predefined time. Simulation results verify the effectiveness of the proposed approach.

Advancements in physiologically based pharmacokinetic modeling for fedratinib: updating dose guidance in the presence of a dual inhibitor of CYP3A4 and CYP2C19.

A physiologically based pharmacokinetic (PBPK) model for fedratinib was updated and revalidated to bridge a gap between the observed drug-drug interaction (DDI) of a single sub-efficacious dose in healthy participants and the potential DDI in patients with cancer at steady state. The study aimed to establish an appropriate dose for fedratinib in patients coadministered with dual CYP3A4 and CYP2C19 inhibitors, providing quantitative evidence to inform dosing guidance. The original minimal PBPK model was developed using Simcyp® Simulator v17. The model was updated by substituting a single distribution rate (Qsac) with 2 separate rates (CLin/CLout) and transitioning to v20. Model parameter updates were further informed with 3 clinical studies, and 3 more studies served as independent validation data. The validated model was applied to simulate potential DDIs between fedratinib and a known dual inhibitor of CYP3A4 and CYP2C19 (fluconazole). Coadministration of fedratinib with fluconazole in patients was predicted to increase fedratinib exposure by < 2-fold in all simulated scenarios. For patients with cancer receiving the approved dose of fedratinib 400mg once daily along with fluconazole 200mg daily, the model predicted an approximate 50% increase in fedratinib exposure at steady state. The updated PBPK model improved description of the observed pharmacokinetics and predicted a low risk of clinically significant DDIs between fedratinib and fluconazole. The quantitative evidence serves as a primary foundation for providing dose guidance in clinical practice for the coadministration of fedratinib with dual CYP3A4 and CYP2C19 inhibitors.

Data-driven parameterization and development of mechanistic cell cultivation models in monoclonal antibody production processes: Shifts in cell metabolic behavior

Representative kinetic models to describe monoclonal antibody (mAb) production processes are needed for effective process design. The development of mechanistic models can be impeded by the lack of complete understanding of changes in cell metabolism, e.g., lactate metabolic shifts. State-estimation-based methods were applied to assess the fit of available kinetic models over experimental runs. The results indicated the regions where model parameter updates were required. Different clustering strategies were applied to isolate the variations in the culture environment and correlate them to the lactate shifts. Alternative formulations for the specific lactate consumption/production term were provided for each identified phase. Two case studies are presented for pilot-scale data in different reactor types. The results show the improvement in modeling accuracy and highlight the role of oxygen and nutrient levels on the shifts. The approach showcases the use of data-driven insights to effectively utilize limited experimental data to develop robust mechanistic models.

Adaptive Visual Control for Robotic Manipulators with Consideration of Rigid-Body Dynamics and Joint-Motor Dynamics

A novel cascade visual control scheme is proposed to tailor for electrically driven robotic manipulators that operate under kinematic and dynamic uncertainties, utilizing an uncalibrated stationary camera. The proposed control approach incorporates adaptive weight radial basis function neural networks (RBFNNs) to learn the behaviors of the uncertain dynamics of the robot and the joint actuators. The controllers are designed to nullify the approximation error and mitigate unknown disturbances through an integrated robust adaptive mechanism. A major advantage of the proposed approach is that prior knowledge of the dynamics of the robotic manipulator and its actuator is no longer required. The controller autonomously assimilates the robot and actuator dynamics online, thereby obviating the need for fussy regression matrix derivation and advance dynamic measurement to establish the adaptive dynamic parameter update algorithm. The proposed scheme ensures closed-loop system stability, bounded system states, and the convergence of tracking errors to zero. Simulation results, employing a PUMA manipulator as a testbed, substantiate the viability of the proposed control policy.

A review of federated learning algorithms in image classification

Image classification is one of the most popular applications of machine learning. It has shown its potential in fields like healthcare, auto-driving and face recognition. Federated learning (FL) emerged in 2017, creating a major innovation to the field. The new structure brings new possibilities, but create new challenges such as data heterogeneity, privacy leakage and communication burdens in parameter updating. The paper solves the problem that there are relatively few papers providing a complete analysis relating to the application of FL in image classification. The paper contributes to describe the new challenges of FL in image classification and show the related reasons behind respectively, then analyze the current state-of-the-art algorithms designed for solving the challenges and improving image model performance by discussing the basic ideas and steps of algorithms and showing their pros and cons. The paper further more contributes to compare the performance of each model in accuracy and communication speed, and outline several possible directions for future advancement.

Research on Real-Time Prediction Method of Photovoltaic Power Time Series Utilizing Improved Grey Wolf Optimization and Long Short-Term Memory Neural Network

This paper proposes a novel method for the real-time prediction of photovoltaic (PV) power output by integrating phase space reconstruction (PSR), improved grey wolf optimization (GWO), and long short-term memory (LSTM) neural networks. The proposed method consists of three main steps. First, historical data are denoised and features are extracted using singular spectrum analysis (SSA) and complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN). Second, improved grey wolf optimization (GWO) is employed to optimize the key parameters of phase space reconstruction (PSR) and long short-term memory (LSTM) neural networks. Third, real-time predictions are made using LSTM neural networks, with dynamic updates of training data and model parameters. Experimental results demonstrate that the proposed method has significant advantages in both prediction accuracy and speed. Specifically, the proposed method achieves a mean absolute percentage error (MAPE) of 3.45%, significantly outperforming traditional machine learning models and other neural network-based approaches. Compared with seven alternative methods, our method improves prediction accuracy by 15% to 25% and computational speed by 20% to 30%. Additionally, the proposed method exhibits excellent prediction stability and adaptability, effectively handling the nonlinear and chaotic characteristics of PV power.

Methods to Quantify the Importance of Parameters for Model Updating and Distributional Adaptation.

Decision models are time-consuming to develop; therefore, adapting previously developed models for new purposes may be advantageous. We provide methods to prioritize efforts to 1) update parameter values in existing models and 2) adapt existing models for distributional cost-effectiveness analysis (DCEA). Methods exist to assess the influence of different input parameters on the results of a decision models, including value of information (VOI) and 1-way sensitivity analysis (OWSA). We apply 1) VOI to prioritize searches for additional information to update parameter values and 2) OWSA to prioritize searches for parameters that may vary by socioeconomic characteristics. We highlight the assumptions required and propose metrics that quantify the extent to which parameters in a model have been updated or adapted. We provide R code to quickly carry out the analysis given inputs from a probabilistic sensitivity analysis (PSA) and demonstrate our methods using an oncology case study. In our case study, updating 2 of 21 probabilistic model parameters addressed 71.5% of the total VOI and updating 3 addressed approximately 100% of the uncertainty. Our proposed approach suggests that these are the 3 parameters that should be prioritized. For model adaptation for DCEA, 46.3% of the total OWSA variation came from a single parameter, while the top 10 input parameters were found to account for more than 95% of the total variation, suggesting efforts should be aimed toward these. These methods offer a systematic approach to guide research efforts in updating models with new data or adapting models to undertake DCEA. The case study demonstrated only very small gains from updating more than 3 parameters or adapting more than 10 parameters. It can require considerable analyst time to search for evidence to update a model or to adapt a model to take account of equity concerns.In this article, we provide a quantitative method to prioritze parameters to 1) update existing models to reflect potential new evidence and 2) adapt existing models to estimate distributional outcomes.We define metrics that quantify the extent to which the parameters in a model have been updated or adapted.We provide R code that can quickly rank parameter importance and calculate quality metrics using only the results of a standard probabilistic sensitivity analysis.

A mission planning method for deep space detectors using deep reinforcement learning

Mission planning for deep space detectors is a pivotal step in the successful execution of detection missions. Traditional planning approaches, which typically compartmentalize mission planning and data transmission scheduling, exhibit limitations in adapting to uncertainties and are often inadequate in responding to unforeseen opportunity targets. This paper introduces a mission planning method for deep space detectors designed to address these challenges. Firstly, a Markov Decision Process (MDP) model is formulated, integrating mission planning and data transmission scheduling, with a consideration for planning balance between detection missions and opportunity targets. Subsequently, a Planning Balance with Proximal Policy Optimization (PB-PPO) algorithm is proposed. The proposed algorithm, based in the Proximal Policy Optimization (PPO) algorithm, integrates an orthogonal initialization algorithm to afford improved control over parameter updates. Furthermore, a dynamic learning rate strategy is implemented to accelerate convergence speed. Experimental results show that, the PB-PPO achieves rewards that are 4.42%, 6.78%, 18.32%, and 26.81% higher than those obtained by compared algorithms. Additionally, the PB-PPO demonstrates the capability to address the planning balance between detection missions and opportunity targets, ensuring stable reward growth even when planning a significant number of opportunity targets. In summary, the PB-PPO integrates of MDP, deep reinforcement learning, and innovative strategies to make it a robust solution for detector mission planning in the complex and dynamic environment of deep space.

Study on the relevant parameters of direct economic loss assessment of reinforced concrete structure buildings considering seismic fortification and use function

The main cost and decoration cost of the building are the key parameters in the evaluation of the direct economic loss of the building. In the Earthquake Site Work Part 4 : Direct Loss Assessment of Disasters ( GB / T 18208.4-2011 ), considering factors such as economic development level, structural type, and building use, based on expert experience and engineering cost data, a grading index was developed and a corresponding correction coefficient was given to more accurately and reasonably assess the loss of the main building and the loss of decoration. In view of the rapid development of China 's economy and the acceleration of urbanization in the past ten years, the original grading indicators and coefficients need to be adjusted to adapt to the reality. This paper focuses on the reinforced concrete structure with various functions and wide distribution. By collecting a large number of corresponding building main body and decoration cost data, and combining with expert experience data, this paper analyzes the influence of seismic fortification level on the main body cost, as well as the influence of economic development level, decoration grade, use function and other factors on the decoration cost. The research results of this paper aim to provide reference for the update and refinement of several key parameters in the specification.

Model Parameter Calibration for Vibration Fatigue Analysis by Means of Bayesian Updating and Artificial Neural Network Based Surrogate Models

Abstract In this paper, a methodology for model parameter calibration for vibration fatigue analysis is proposed. It combines Bayesian updating of uncertain model parameters and artificial neural networks (ANNs). The calibrated parameters are used to increase the accuracy of fatigue lifetime calculations for components submitted to vibrational loads. The Bayesian updating uses eigenfrequencies, mode shapes, total mass, and the frequency response functions (FRFs). These quantities are predicted by ANN-based surrogate models to accelerate the Bayesian updating process. A novel strategy for the prediction of the magnitude and phase of FRFs with ANNs is proposed. The frequency is used as an additional input variable, and a schematic selection of significant points of the FRF curves is presented. A high prediction accuracy of the surrogate models could be achieved. The procedure includes the analysis of the relevant frequency range and a sensitivity analysis based on the Morris method to identify appropriate modes and the most-influential parameters. The proposed framework is applied to a current vehicle component subjected to vibrational loads. An experimental modal analysis is used for the calibration and consideration of real parameter uncertainty. First, the accuracy of the surrogate models and Bayesian updating is verified by a nominal reference simulation and then validated with experimental data. The measurable control parameter thickness and component mass are used to examine the calibration accuracy. Finally, a decrease in the dispersion of the vibration fatigue distribution is obtained with the calibrated parameters.

Structural prior-driven feature extraction with gradient-momentum combined optimization for convolutional neural network image classification

Recent image classification efforts have achieved certain success by incorporating prior information such as labels and logical rules to learn discriminative features. However, these methods overlook the variability of features, resulting in feature inconsistency and fluctuations in model parameter updates, which further contribute to decreased image classification accuracy and model instability. To address this issue, this paper proposes a novel method combining structural prior-driven feature extraction with gradient-momentum (SPGM), from the perspectives of consistent feature learning and precise parameter updates, to enhance the accuracy and stability of image classification. Specifically, SPGM leverages a structural prior-driven feature extraction (SPFE) approach to calculate gradients of multi-level features and original images to construct structural information, which is then transformed into prior knowledge to drive the network to learn features consistent with the original images. Additionally, an optimization strategy integrating gradients and momentum (GMO) is introduced, dynamically adjusting the direction and step size of parameter updates based on the angle and norm of the sum of gradients and momentum, enabling precise model parameter updates. Extensive experiments on CIFAR10 and CIFAR100 datasets demonstrate that the SPGM method significantly reduces the top-1 error rate in image classification, enhances the classification performance, and outperforms state-of-the-art methods.

Robust and Privacy-Preserving Decentralized Deep Federated Learning Training: Focusing on Digital Healthcare Applications.

Federated learning of deep neural networks has emerged as an evolving paradigm for distributed machine learning, gaining widespread attention due to its ability to update parameters without collecting raw data from users, especially in digital healthcare applications. However, the traditional centralized architecture of federated learning suffers from several problems (e.g., single point of failure, communication bottlenecks, etc.), especially malicious servers inferring gradients and causing gradient leakage. To tackle the above issues, we propose a robust and privacy-preserving decentralized deep federated learning (RPDFL) training scheme. Specifically, we design a novel ring FL structure and a Ring-Allreduce-based data sharing scheme to improve the communication efficiency in RPDFL training. Furthermore, we improve the process of distributing parameters of the Chinese residual theorem to update the execution process of the threshold secret sharing, supporting healthcare edge to drop out during the training process without causing data leakage, and ensuring the robustness of the RPDFL training under the Ring-Allreduce-based data sharing scheme. Security analysis indicates that RPDFL is provable secure. Experiment results show that RPDFL is significantly superior to standard FL methods in terms of model accuracy and convergence, and is suitable for digital healthcare applications.

Co-Frequency Interference Suppression of Integrated Detection and Jamming System Based on 2D Sparse Recovery

The integrated detection and jamming system employs integrated signals devoid of typical radar signal characteristics for detection and jamming. This allows for the sharing of resources such as waveform, frequency, time, and aperture, significantly enhancing the overall utilization rate of system resources. However, to achieve effective interference, the integrated waveform must overlap with the adversary radar signal within the frequency band. Consequently, the detection echoes are susceptible to the strong co-frequency direct wave generated by the adversary signals. This paper proposes a co-frequency direct wave interference suppression algorithm based on 2D generalized smoothed-l0 norm sparse recovery. The algorithm exploits a joint dictionary comprising our integrated signals and adversary signals, along with the sparsity of 2D range-Doppler maps. The direct solution of the sparse decomposition optimization problem, formulated for the entire echo matrix, enhances the target detection performance for integrated signals even in the presence of robust co-frequency direct wave interference. Furthermore, the proposed method achieves robustness to interference of varying intensities through the adaptive updating and adjustment of relevant parameters. The effectiveness of the proposed method is validated through simulation and experimental results.

Probabilistic fatigue life prediction using multi-layer perceptron with maximum entropy algorithm

In the structural fatigue fields, such as aerospace, maritime, and civil engineering, the accurate construction of the Probability-Stress-Life (P-S-N) curve is crucial for preventing unexpected structural failures and is fundamental to the engineering design process. Conventional methods usually assume that the fatigue life distribution follows a normal or weibull distribution, which overlooks the differences of life distribution at different stress levels and introduces subjective errors. To address this issue, this paper proposes a new approach based a multi-layer perceptron model with maximum entropy algorithm (MPME) for probabilistic fatigue life prediction. During the derivation of life probability distribution function in this approach, the type not require any pre-assumption, thus effectively avoiding the introduction of subjective errors. Specifically, the dispersed information of fatigue life is quantified as various life distribution statistical features such as mean, variance, skewness, and kurtosis. Next, learn the continuous functional relationship between stress and these features using a multi-layer perceptron (MLP). Finally, the target probability distribution is derived using the maximum entropy algorithm, with the predicted distribution features obtained from the MLP serving as constraints for it. In addition, to ensure the consistency between the prediction of MLP and physical laws, we encode physical knowledge into the network loss to guide the updating of network parameters. Experimental results of two different material fatigue data demonstrate that, compared to traditional deterministic model-based methods and machine learning methods, the MPME in this paper can more accurately predict the probabilistic fatigue life.

MDTGAN: Multi domain generative adversarial transfer learning network for traffic data imputation

Accurate, real-time, and efficient traffic data, crucial for intelligent transportation systems, which is often disrupted in the real world due to the influence of weather, interference, and equipment failures. Generative Adversarial Networks (GANs), have achieved significant results in image restoration, providing insights for traffic data imputation. Recent research has focused on developing more effective and reasonable model by learning transferable common knowledge from different cities or different scenarios, which is also an excellent idea for improving the data imputation performance. In light of, we contrive to design a GANs based transferable traffic data imputation model, namely Multi Domain Generative Adversarial Transfer Learning Network (MDTGAN). Firstly, the model consists of two stages. In pre-training stage, the model utilizes the source domain datasets (Similar cities road network datasets) to update parameters and transferred them. Then, the model parameters are optimized using the target domain dataset (Target city road network dataset) in fine-tuning stage to avoid the issue of insufficient samples caused by data missing. Secondly, to adapt the model to different dataset topologies, MDTGAN models the node-level data by taking node time series as input, enabling the model to autonomously learn the prevalent spatiotemporal correlations inherent in nodes. Additionally, we introduce a domain discriminator module that guides the spatial encoder in learning domain-invariant features of network node spatial information, enhancing the model generalization ability. The experiments conducted on three publicly available datasets, in which one dataset is regarded as the target dataset, while the others serve as source datasets. The experimental results demonstrate that the MDTGAN model consistently outperforms the baseline models. Specifically, the MDTGAN model can transfer valuable knowledge from the source domain datasets to improve data imputation performance on the target domain dataset, providing practical significance for cities lacking historical traffic data.

Few-shot online anomaly detection and segmentation

Detecting anomaly patterns from images is a crucial artificial intelligence technique in industrial applications. Recent research in this domain has emphasized the necessity of a large volume of training data, overlooking the practical scenario where, post-deployment of the model, unlabeled data containing both normal and abnormal samples can be utilized to enhance the performance. Consequently, this paper focuses on addressing the challenging yet practical few-shot online anomaly detection and segmentation (FOADS) task. Under the FOADS framework, models are trained on a few-shot normal dataset, followed by inspection and improvement of their capabilities by leveraging unlabeled streaming data containing both normal and abnormal samples simultaneously. Since the data stream has no ground truth labels, the model needs to filter the anomalous data by detection to avoid contaminating its parameters, which presents a significant challenge when initial samples are insufficient. To tackle this issue, we propose modeling the feature distribution of normal images using a Neural Gas network, which offers the flexibility to adapt the topology structure to identify outliers in the data flow. In order to achieve improved performance with limited training samples, we employ multi-scale feature embedding extracted from a CNN pre-trained on ImageNet to obtain a robust representation. Furthermore, we introduce an algorithm that can incrementally update parameters without the need to store previous samples. Comprehensive experimental results demonstrate that our method can achieve substantial performance under the FOADS setting, while ensuring that the time complexity remains within an acceptable range on MVTec AD and BTAD datasets.