Critical Network Research Articles

An efficient and lightweight off-policy actor–critic reinforcement learning framework

In the framework of current off-policy actor–critic methods, the state–action pairs in an experience replay buffer (called historical behaviors) cannot be used to improve the policy, and the target network and the clipped double Q-learning techniques need to be used to evaluate the policy. The framework limits the policy learning capability in complex environments, and needs to maintain four critic networks. As a result, we propose an efficient and lightweight off-policy actor–critic (EL-AC) framework. In the policy improvement, an efficient off-policy likelihood ratio policy gradient algorithm with historical behaviors reusing (PG-HBR) is proposed, which promotes the agent to learn an approximately optimal policy by using the historical behaviors. Moreover, a theoretically interpretable universal critic network is designed. It can approximate the action-value and the state-value functions simultaneously, so as to obtain the advantage function in PG-HBR. In the policy evaluation, we develop the algorithms of low-pass filtering for target state-values and adaptive controlling algorithm for overestimation bias, which can evaluate the policy efficiently and accurately using only one universal critic network. Extensive evaluation results indicate that EL-AC outperforms the state-of-the-art off-policy actor–critic methods in terms of approximately optimal policy learning and neural network storage space occupation, and it is more suitable for policy learning in complex environments.

Critical Information Mining Network: Identifying Crop Diseases in Noisy Environments

When agricultural experts explore the use of artificial intelligence technology to identify and detect crop diseases, they mainly focus on the research of a stable environment, but ignore the problem of noise in the process of image acquisition in real situations. To solve this problem, we propose an innovative solution called the Critical Information Mining Network (CIMNet). Compared with traditional models, CIMNet has higher recognition accuracy and wider application scenarios. The network has a good effect on crop disease recognition under noisy environments, and can effectively deal with the interference of noise to the recognition effect in actual farmland scenes. Consider that the shape of the leaves can be symmetrical or asymmetrical.First, we introduce the Non-Local Attention Module (Non-Local), which uses a unique self-attention mechanism to fully capture the context information of the image. The module overcomes the limitation of traditional convolutional neural networks that only rely on local features and ignore global features. Global features are particularly important when the image is disturbed by noise. Non-Local improves a more comprehensive visual understanding of crop disease recognition. Secondly, we have innovatively designed a Multi-scale Critical Information Fusion Module (MSCM). The module uses the Key Information Extraction Module (KIB) to dig into the shallow key features in the network deeply. The shallow key features strengthen the feature perception of the model to the noise image through texture and contour information, and then the shallow key features and deep features are fused to enrich the original deep feature information of the network. Finally, we conducted experiments on two public datasets, and the results showed that the accuracy of our model in crop disease identification under a noisy environment was significantly improved. At the same time, our model also showed excellent performance under stable conditions. The results of this study provide favorable support for the improvement of crop production efficiency.

Assessment of residual prestress in existing concrete bridges: The Kalix bridge

The direct socio-economic consequences of the deterioration of aging infrastructure systems have triggered a continuous process of revising and updating current design standards and guidelines for critical network components. Specifically, long-term degradation processes demand the analysis and evaluation of vital structural assets such as prestressed concrete bridges. It is crucial to develop theoretically consistent, user-friendly, and non-destructive methodologies that engineering professionals can employ to prevent and mitigate potential catastrophic outcomes during the service life of these bridges. This study provides a thorough review of the available testing methods employed over the years for prestressed concrete bridges and introduces a comprehensive framework for evaluating existing methods for residual prestress force assessment. Through a multi-criteria selection process, the three most feasible tests were designed and carried out on an existing 66-year-old balanced cantilever box girder bridge exposed to freezing temperatures that affected the instrumentation plan and test execution. Finally, predictive models compliant with standard codes were calibrated based on the experimental results and the life cycle loss of prestress forces was evaluated to assess relevant bounding intervals. Findings reveal limited on-site testing and discrepancies between calculated residual forces and predictions by standard codes. The saw cut method showed a 18% difference from the initial applied prestress according to the prestress protocol, suggesting the use of a cover meter and concrete modulus evaluation for improved accuracy. The strand cutting method resulted in a 14% difference, emphasizing the need for stress redistribution assessment. The second-order deflection method showed a 6% difference, indicating a focus on enhanced boundary conditions and thorough sensitivity analysis for future investigations.

Backstepping based neural [formula omitted] optimal tracking control for nonlinear state constrained systems with input delay and disturbances

This paper focuses on the problem of H∞ optimal tracking control for a class of nonlinear state constrained systems with input delay and disturbances. With the aid of Pade approximation, an auxiliary variable is devised to eliminate the effects of input delay. Combining barrier Lyapunov functions (BLFs) with backstepping design technique, a feedforward adaptive controller is designed to transform the tracking control problem of nonlinear state constrained system into an equivalent H∞ control problem of input-affine error system without state constraints, wherein neural networks (NNs) are employed to approximate unknown system dynamics. Then based on single-network adaptive dynamic programming (ADP), an H∞ optimal feedback controller is developed by utilizing a single critic network to learn the Nash equilibrium related to Hamilton–Jacobi–Isaacs (HJI) equation. Therefore, the whole tracking controller can be constructed by integrating the feedforward adaptive controller with the optimal feedback controller. Moreover, it is proven by Lyapunov’s theory that all signals within the closed-loop system are uniformly ultimately bounded (UUB), and the tracking error converges to a small neighborhood of the origin without violating any state constraints. Two simulation examples are also presented to validate the effectiveness of the proposed approach.

Critic learning control via zero-sum differential game for affine formation maneuver of multi-agent systems with cyber-attacks

The affine formation maneuver (AFM) control problem using the zero-sum differential game is studied for a second-order multi-agent system (MAS) against cyber-injection attacks on the actuators. First and foremost, the distributed control signals and the cyber-injected attack signals are viewed as two competing teams. Then, the leader-follower tracking error system is established where the followers aim to track the AFMs of the leaders. The leader-follower distributed optimal control policies and the cyber-attack attenuation policies are derived from the Nash equilibrium solution of the Hamilton-Jacobi-Isaac (HJI) equation. The solution of the HJI equation is estimated with the aid of the critic neural network. The weights of the critic system are learned online based on the adaptive dynamic programming (ADP) scheme. The uniformly ultimately bounded stability of the closed-loop system under the formulated zero-sum differential game control strategy has been proven by the Lyapunov stability theorem. A simulation example illustrates that the presented control method can attenuate cyber-injection attacks on the actuators and maintain the AFMs of the agents.

Real-time fusion multi-tier DNN-based collaborative IDPS with complementary features for secure UAV-enabled 6G networks

UAV-enabled Integrated Sensing and Communication (ISAC) in sixth-generation (6G) wireless networks has sparked significant research interest. UAVs are positioned as aerial wireless platforms, extending coverage and improving Sensing and Communication (S&C) services. However, integrating UAVs introduces vulnerabilities due to multi-connectivity, necessitating robust security measures. To address this, efforts are focused on developing effective Intrusion Detection Systems (IDS). Here ML-based IDSs depend on the most suited Machine Learning (ML) algorithms for improved detection accuracy. However, inadequate detection features frequently contribute to the limitations of detection accuracy in various emerging cyber-attacks. Our study explores attack traits in UAV-enabled 6G networks, using complementary detection features to enhance ML-based attack detection. Additionally, existing Deep Neural Network (DNN) solutions for UAVs-enabled 6G networks mainly use single models (e.g., CNN, SVM, CGAN) instead of multi-tier models, (e.g., Hybrid, Fusion, Ensemble). Here, although the use of a single model can be faster for computer systems, it is difficult to understand the growing complexity of intrusion patterns in data. Also, these single models might not be good at recognizing the unique patterns of less common issues in datasets. Thus, we propose a fusion multi-tier DNN-based Collaborative Intrusion Detection and Prevention System (CIDPS) for critical UAV-enabled 6G networks. This system boosts accuracy without sacrificing latency reductions. Our approach learns decision boundaries from imbalanced data points using preceding DNNs sequentially. Moreover, most existing research works do not focus on intrusion prevention methods and deployment frameworks for UAV networks. This research proposes effective IPS mechanisms and deployment architecture for CIDPS in UAV-enabled 6G networks. It incorporates a CIDPS with an emergency response protocol, neutralizing attacks upon anomaly detection. We validate our solution through diverse dataset experiments (UAVIDS-2020, NF-UQ-NIDS-v2, 5G-NIDD). Unlike traditional practices where IDS are simulated, we implement CIDPS on actual UAV devices (PX4 Vision Dev Kit V1.5, DJI mini se, DJI mini 3 pro) and real-world UAV networks. Our approach outperforms prior algorithms with a 99.25% attack classification accuracy, higher detection efficiency, and lower resource usage, exceeding 99.05% detection rates.

Higher mortality among off-hours versus regular-hours admissions to the cardiac intensive care unit (CICU): insights from the critical care cardiology trials network registry (CCCTN)

Abstract Funding Acknowledgements None. Background The outcomes of patients admitted to general intensive care units vary significantly according to the time of admission. Whether this applies to the CICU setting is unknown. Purpose We sought to determine if patient characteristics, admission source, critical care therapy utilization, and clinical outcomes differ according to whether CICU admission occurred during regular-hours or off-hours. Methods CCCTN is a multicenter prospective registry of consecutive admissions to CICUs in North America coordinated by the TIMI Study Group. For this analysis of admissions during calendar year 2022, "regular-hours" was defined as 7 a.m.- 7 p.m. Monday-Friday, and "off-hours" was defined as 7 p.m.- 7 a.m. on weekdays or at any time on Saturday or Sunday. Logistic regression analysis was used to adjust in-hospital mortality rates for admission time (on-hours vs. off-hours), age, sex, SOFA score, and cardiac arrest. Results Among 5238 CICU admissions, 2956 (56.4%) occurred during off-hours. Patient demographics (mean age: 67.0 years, 35.7% female, 51.1% white) and comorbidities were generally similar among off-hours and regular-hours admissions. The median SOFA score was 3.0 in both groups. Compared to regular-hours admissions, off-hours admissions were more likely to originate from the ED (39.9% vs. 29.4%, p<0.0001) and less likely to originate from a procedural area (24.5% vs. 41.8%, p<0.0001). ST-elevation MI was a more common admission diagnosis among off-hours vs. regular-hours admissions (19.2% vs. 16.4%, p=0.009). Primary CICU indications of respiratory insufficiency, unstable arrhythmia, cardiac arrest, and distributive/mixed shock were more common among off-hours admissions (Table). Mechanical ventilation, IV vasoactive therapy, and targeted temperature management (TTM) were utilized more frequently among patients admitted during off-hours. Median ICU (2.5 vs. 2.0d) and hospital length of stay (6.2 vs. 5.8d) were longer for off-hours admissions (both p<0.0001). Crude CICU mortality (11.4% vs. 8.5%, p=0.0005) and in-hospital mortality (15.6% vs. 11.7%, p<0.0001) were significantly higher among off-hours admissions (Figure). After adjustment, off-hours admission remained significantly associated with higher in-hospital mortality (OR 1.2, 95% CI 1.02 – 1.49, p=0.03). Conclusions Off-hours and regular-hours admissions to the CICU had similar baseline characteristics but differed significantly regarding source of admission, indication for admission, and utilization of critical care therapies. These differences were associated with worse clinical outcomes among off-hours admissions, which may suggest the need for more intensive staffing in CICUs during these times.FigureTable

Mycobactin analogue interacting with siderophore efflux-pump protein: insights from molecular dynamics simulations and whole-cell assays

IntroductionIn response to continued public health emergency of antimicrobial resistance (AMR), a significant key strategy is the discovery of novel mycobacterial efflux-pump inhibitors (EPIs) as potential adjuvants in combination drug therapy. Interest in identifying new chemotypes which could potentially synergize with the existing antibiotics and can be deployed as part of a combination therapy. This strategy could delay the emergence of resistance to existing antibiotics and increase their efficacy against resistant strains of mycobacterial species. In recent decades, notable approaches have been accounted for EPI development and have resulted in the discovery of several EPIs including SQ109 and AU1235. In context, to accelerate newer EPIs with novel mode of action here we have discussed mycobactin analogues and highlighted in silico binding orientation with siderophore efflux-pump proteins MmpL4/5.Methods3-(2-hydroxyphenyl)-5-(aryl)-pyrazoline series was investigated for whole-cell efflux-pump inhibitory activity against Mycobacterium smegmatis and Mycobacterium abscessus. Machine learning and molecular dynamics were performed to construct a MmpL4/5 complex embedded in a lipid bilayer to identify the putative binding site and to predict ligand-protein binding energetics. Furthermore, the identified HIT compound was investigated in synergistic assay with bedaquiline.ResultsCompound Il, 2-(5-(4-fluorophenyl)-4,5-dihydro-1H-pyrazol-3-yl)phenol, was identified as the most potent efflux pump inhibitor against M. smegmatis in whole-cell efflux-pump investigation. Followed HIT Il employed against M. abscessus for efflux-pump inhibition investigations and notable whole-cell efflux-pump inhibitory profile has been observed. The theoretical investigations predicted compound Il to be selective towards MmpL4, with significant hydrogen bonding and π-π stacking interactions effectively blocking a critical Asp-Tyr dyad interaction network necessary for proton translocation. Compound Il with bedaquiline highlighted an additive profile against the M. abscessus pathogen.ConclusionsMD simulations and whole-cell assays are indicating potential development of compound Il as an adjunct to the existing therapeutic regimen against mycobacterial infections.

Neural critic learning with accelerated value iteration for nonlinear model predictive control

In practical industrial processes, the receding optimization solution of nonlinear model predictive control (NMPC) is always a very knotty problem. Based on adaptive dynamic programming, the accelerated value iteration predictive control (AVI-PC) algorithm is developed in this paper. Integrating iteration learning with the receding horizon mechanism of NMPC, a novel receding optimization solution pattern is exploited to resolve the optimal control law in each prediction horizon. Besides, the basic architecture and the specific form of the AVI-PC algorithm are demonstrated, including the relationship among the iterative learning process, the prediction process, and the control process. On this basis, the convergence and admissibility conditions are established, and the relevant properties are comprehensively analyzed when the accelerated factor satisfies the established conditions. Furthermore, the accelerated value iterative function is approximated through the single critic network constructed by utilizing the multiple linear regression method. Finally, the plentiful simulation experiments are conducted from various perspectives to verify the effectiveness and progressiveness of the AVI-PC algorithm.

Adaptive Critic Learning-Based Optimal Bipartite Consensus for Multiagent Systems With Prescribed Performance.

Developing a distributed bipartite optimal consensus scheme while ensuring user-predefined performance is essential in practical applications. Existing approaches to this problem typically require a complex controller structure due to adopting an identifier-actor-critic framework and prescribed performance cannot be guaranteed. In this work, an adaptive critic learning (ACL)-based optimal bipartite consensus scheme is developed to bridge the gap. A newly designed error scaling function, which defines the user-predefined settling time and steady accuracy without relying on the initial conditions, is then integrated into a cost function. The backstepping framework combines the ACL and integral reinforcement learning (IRL) algorithm to develop the adaptive optimal bipartite consensus scheme, which contributes a critic-only controller structure by removing the identifier and actor networks in the existing methods. The adaptive law of the critic network is derived by the gradient descent algorithm and experience replay to minimize the IRL-based residual error. It is shown that a compute-saving learning mechanism can achieve the optimal consensus, and the error variables of the closed-loop system are uniformly ultimately bounded (UUB). Besides, in any bounded initial condition, the evolution of bipartite consensus is limited to a user-prescribed boundary under bounded initial conditions. The illustrative simulation results validate the efficacy of the approach.

Critical complex network structures in animal gastrointestinal tract microbiomes

BackgroundLiving things from microbes to their hosts (plants, animals and humans) interact with each other, and their relationships may be described with complex network models. The present study focuses on the critical network structures, specifically the core/periphery nodes and backbones (paths of high-salience skeletons) in animal gastrointestinal microbiomes (AGMs) networks. The core/periphery network (CPN) mirrors nearly ubiquitous nestedness in ecological communities, particularly dividing the network as densely interconnected core-species and periphery-species that only sparsely linked to the core. Complementarily, the high-salience skeleton network (HSN) mirrors the pervasive asymmetrical species interactions (strictly microbial species correlations), particularly forming heterogenous pathways in AGM networks with both “backbones” and “rural roads” (regular or weak links). While the cores and backbones can act as critical functional structures, the periphery nodes and weak links may stabilize network functionalities through redundancy.ResultsHere, we build and analyze 36 pairs of CPN/HSN for the AGMs based on 4903 gastrointestinal-microbiome samples containing 473,359 microbial species collected from 318 animal species covering all vertebrate and four major invertebrate classes. The network analyses were performed at host species, order, class, phylum, kingdom scales and diet types with selected and comparative taxon pairs. Besides diet types, the influence of host phylogeny, measured with phylogenetic (evolutionary) timeline or “age”, were integrated into the analyses. For example, it was found that the evolutionary trends of three primary microbial phyla (Bacteroidetes/Firmicutes/Proteobacteria) and their pairwise abundance-ratios in animals do not mirror the patterns in modern humans phylogenetically, although they are consistent in terms of diet types.ConclusionsOverall, the critical network structures of AGMs are qualitatively and structurally similar to those of the human gut microbiomes. Nevertheless, it appears that the critical composition (the three phyla of Bacteroidetes, Firmicutes, and Proteobacteria) in human gut microbiomes has broken the evolutionary trend from animals to humans, possibly attributable to the Anthropocene epoch and reflecting the far-reaching influences of agriculture and industrial revolution on the human gut microbiomes. The influences may have led to the deviations between modern humans and our hunter-gather ancestors and animals.

A Hybrid Controller for Musculoskeletal Robots Targeting Lifting Tasks in Industrial Metaverse.

In manufacturing, musculoskeletal robots have gained more attention with the potential advantages of flexibility, robustness, and adaptability over conventional serial-link rigid robots. Focusing on the fundamental lifting tasks, a hybrid controller is proposed to overcome control challenges of such robots for widely applications in industry. The metaverse technology offers an available simulated-reality-based platform to verify the proposed method. The hybrid controller contains two main parts. A muscle-synergy-based radial basis function (RBF) network is proposed as the feedforward controller, which is able to characterize the phasic and the tonic muscle synergies simultaneously. The adaptive dynamic programming (ADP) is applied as the feedback controller to address the optimal control problem. The actor-critic structure is applied in the ADP-based controller, where the critic network is trained to approximate the optimal performance index and the actor network is trained to compute the optimal muscle excitations. Furthermore, the convergence and stability of the ADP algorithm are also analyzed. Finally, experiments have been designed to verify the effectiveness of this hybrid controller on an upper limb musculoskeletal system, and the comparisons with other controllers are also illustrated. The results show that the proposed controller can obtain a satisfactory performance for lifting tasks.

Optimizing resource allocation in UAV-assisted ultra-dense networks for enhanced performance and security

The deployment of unmanned aerial vehicles (UAVs) in ultra-dense networks (UNDs) has significantly advanced network capabilities in 5G/6G environments, addressing coverage enhancement and security concerns. Our research presents a deep reinforcement learning (DRL) based approach designed to manage the increasing data traffic demands and limited communication resources in UAV-assisted UNDs. Traditional DRL methodologies often struggle with challenges like low sample efficiency and energy wastage, which can indirectly impact network security and stability. To address these concerns, we introduce the Stabilizing Transformers based Potential Driven Reinforcement Learning (STPD-RL) framework. STPD-RL optimizes critical network operations such as transmission link selection and power allocation, directly contributing to improved energy efficiency and robust network performance. Initially, we have refined the potential driven experience replay and implemented it into resource allocation in UAV-assisted UDN for the inaugural time. By assigning a potential energy function to each state in experience replay, users can employ intrinsic state supervision to learn from a spectrum of good and bad experiences. Subsequently, we have employed stabilizing transformers to hasten the learning trajectory for resource allocation policies, thereby enhancing the stability of model training. Furthermore, we have integrated potential driven experience replay and stabilizing transformers within the Proximal Policy Optimization algorithm, thus formulating our uniquely tailored STPD-PPO. In simulations with many users and base stations, STPD-PPO outperformed traditional PPO in metrics such as entropy loss, policy loss, and value loss. Results suggest that our STPD-PPO surpasses traditional DRL algorithms in several respects, including convergence rate, energy efficiency, total power consumption, and exploration capacity.

Adaptive Double-Spending Attacks on PoW-Based Blockchains

Previous state-of-the-art studies have proposed various analytical models to understand the double-spending attacks (DSA) occurred in Proof-of-Work (PoW) based Blockchains. Although many insights behind the double-spending attacks have been disclosed, we still believe that advanced versions of DSA can be developed to create new threats for the PoW-based blockchains such as the Bitcoin blockchain. In this paper, we present two new types of double-spending attacks in the context of the PoW-based blockchain and discuss the insights behind them. By considering more practical network parameters, such as the number of confirmation blocks, the hashpower of the double-spending attacker, the amount of coins in the target transaction, and the network-status parameter, we first analyze the success probability of the conventional double-spending attack, named Naive DSA. Based on Naive DSA, we create two new adaptive DSA, i.e., the Adaptive DSA and the Reinforcement Adaptive DSA (RA-DSA). In our analytical models, a double-spending attack is converted into a Markov Decision Process. We then exploit the Stochastic Dynamic Programming (SDP) approach to obtain the optimal attack strategies under Adaptive DSA and RA-DSA. Numerical simulation results demonstrate the correlations between each critical network parameter and the expected attacker's reward. Through the proposed analytical models, we aim to alert the PoW-based blockchain ecosystem that the threat of double-spending attacks is still at a dangerous level. For example, our findings show that the attacker can launch a successful attack with a small hashpower proportion much lower than 51% under RA-DSA.

Small RNA sequencing highlights a potential regulatory network mediated by Gecko miRNA affecting the prognosis of hepatocellular carcinoma.

Gecko has been widely documented in Chinese scientific literature as an anti-tumor agent for various illnesses for thousands of years, and more recently, it has been examined for its anti-tumor effects on several cancers. The effect of Gecko microRNAs (miRNAs) on hepatocellular carcinoma (HCC) has not yet been reported. This study was designed to identify miRNAs in Gecko through small RNA sequencing and utilize bioinformatics techniques to construct a potential regulatory network and explore the possible mechanisms of exogenous miRNAs involved in HCC. RNA was extracted from Gecko tablets, and we screened the Gecko miRNA expression dataset after high-throughput sequencing. Bioinformatics analysis was used to identify novel Gecko and HCC survival-related miRNA-mRNA cross-species regulation networks. miR-100-5p, miR-99a-5p and miR-101-3p were identified as critical for the role of Geckos in HCC. Nine downstream mRNAs (EZH2, KPNA2, LMNB1, LRRC1, MRGBP, SMARCD1, STMN1, SUB1, and UBE2A) were identified as target genes for critical miRNAs. A miRNA-mRNA regulatory network was constructed, and Gene Ontology (GO) and Kyoto Encyclopedia of Genes and Genomes (KEGG) enrichment analysis showed these key mRNAs might be associated with both the suppression and progression of HCC. The novel network significantly correlated with the abundance of multiple immune cells, as determined with immune infiltration analysis. These findings suggest that Gecko may inhibit progression and exert a therapeutic effect on HCC by targeting critical miRNA-mRNA networks for cross-species regulation. It also provides a reference for future research and development of traditional Chinese medicine (TCM).

Reinforcement learning-based SDN routing scheme empowered by causality detection and GNN.

In recent years, with the rapid development of network applications and the increasing demand for high-quality network service, quality-of-service (QoS) routing has emerged as a critical network technology. The application of machine learning techniques, particularly reinforcement learning and graph neural network, has garnered significant attention in addressing this problem. However, existing reinforcement learning methods lack research on the causal impact of agent actions on the interactive environment, and graph neural network fail to effectively represent link features, which are pivotal for routing optimization. Therefore, this study quantifies the causal influence between the intelligent agent and the interactive environment based on causal inference techniques, aiming to guide the intelligent agent in improving the efficiency of exploring the action space. Simultaneously, graph neural network is employed to embed node and link features, and a reward function is designed that comprehensively considers network performance metrics and causality relevance. A centralized reinforcement learning method is proposed to effectively achieve QoS-aware routing in Software-Defined Networking (SDN). Finally, experiments are conducted in a network simulation environment, and metrics such as packet loss, delay, and throughput all outperform the baseline.

Optimizing anti-collision strategy for MASS: A safe reinforcement learning approach to improve maritime traffic safety

Maritime autonomous surface ships (MASS) promise enhanced efficiency, reduced human errors, and to improve maritime traffic safety. However, MASS navigation in complex maritime traffic presents challenges, especially in collision avoidance strategy optimization (CASO). This paper proposes a novel risk-based CASO approach based on safe reinforcement learning (SRL) with a reliability and risk hierarchical critic network (SRL-R2HCN) approach. Key steps in developing the approach start with the formulation of collision risk assessment. This is followed by the construction of a hierarchical network structure, supplemented by the supporting reward function, multi-objective function, and reliability measurement to realize the SRL-R2HCN. Finally, simulation experiments are conducted in mixed obstacle scenarios, and the results are compared with traditional algorithms to showcase the advancement and fidelity of the new SRL-R2HCN method. The results demonstrate that the proposed method can accurately assess collision risks in mixed obstacle scenarios and generate safe, efficient, and reliable collision avoidance strategies. The outcomes of this research provide a sound theoretical basis for the future development of MASS navigation safety and significant potential to improve the safe and efficient operations of MASS. Furthermore, the methodology could also benefit maritime transportation and shipping management.

Adaptive neural event‐triggered near‐optimal control for affined uncertain nonlinear discrete‐time system

AbstractA novel event‐triggered heuristic dynamic programming (HDP) algorithm is proposed for the near‐optimal control of uncertain discrete‐time nonlinear input‐affine systems. Based on input‐to‐state stability (ISS) analysis, a new event‐triggered mechanism (ETM) is designed. Under constant coefficients, a Lipschitz‐like assumption that forms the basis of the event‐triggering condition is considered to be conservative. To further reduce the conservativeness of the triggering condition and enlarge the average interevent time, an adaptive threshold parameter is utilized in the proposed ETM. In the HDP algorithm framework, model, critic, and action network are adopted to achieve state estimation, approximation to the optimal cost function, and solution to Hamilton–Jacobian–Bellman (HJB) equation. Under the proposed event‐triggered HDP algorithm, the closed system is proved to possess semiglobal uniform ultimate boundedness (SGUUB). Finally, by conducting simulation, it shows that on the premise of satisfying control performance, the event‐triggered strategy can realize reduction on the updating frequency of the controller.

Resilience-based post-earthquake restoration scheduling for urban interdependent transportation-electric power network

As critical lifeline systems, transportation network (TN) and electric power network (EPN) are highly susceptible to natural hazards, such as earthquakes during their service life. At the same time, restoration of damaged TN and EPN is essential to support the post-earthquake reconstruction and emergency rescue in affected areas. Restoration strategies were traditionally developed for TN or EPN separately. However, neglecting the potential interconnection between these two networks in the recovery phase may lead to detrimental consequences, as in real-world scenarios, the obtained strategy may be less efficient or even unfeasible given that recovery of one system is usually dependent on the others for service provision. Accordingly, this paper presents a resilience-based framework for post-earthquake restoration of interdependent transportation-electric power networks. In this framework, restoration independencies and functionality dependencies are introduced to represent the interaction between TN and EPN. Then, a bi-level optimization model with the objective of maximizing seismic resilience is established to characterize the network recovery problem. Furthermore, a solution algorithm that incorporates a genetic algorithm and a chromosome validity test operator is designed to obtain the near-optimal solution. Finally, the proposed framework is illustrated through two numerical examples.

Reinforcement learning-based moving-target enclosing control for an unmanned surface vehicle in multi-obstacle environments

This paper investigates the moving-target enclosing problem in multi-obstacle environments for an unmanned surface vehicle (USV) with complex unknown factors, including target dynamics, vehicle dynamics, and disturbances. A reinforcement learning-based moving-target enclosing control scheme is proposed to ensure collision-free behavior and bolster the enclosing capability. Specifically, an extended state observer is deployed to estimate the target dynamics. Leveraging the estimated data and control obstacle functions, a virtual safety control law is formulated to dynamically harmonize obstacle avoidance and target enclosing control. Then, a novel controller is constructed to track this control law utilizing a Nussbaum-type function in conjunction with actor–critic neural networks (NNs). The actor and critic NNs are employed to approximate unknown dynamics encapsulating vehicle dynamics and disturbances, and value function, respectively. The Nussbaum-type function is embedded to adaptively identify unknown inertia mass, regulating the control input of the USV with the actor NN online. The proposed scheme effectively decouples obstacle avoidance and target enclosing control and only relies on the measurable variables. A rigorous theoretical analysis is further employed to ensure the closed-loop stability of the USV system. Eventually, Simulations are demonstrated to validate the effectiveness and superiority of the proposed scheme for the USV in multi-obstacle environments.