Deep Reinforcement Learning Algorithm Research Articles

Multi-agent cooperative optimal scheduling strategy of integrated energy system in urban area under extreme events

Because the global climate change intensifies as well as the natural disasters frequently occur, extreme events have caused serious impacts on the energy system in urban areas, and at the same time, they have brought great challenges to the supply and scheduling of urban energy systems. Therefore, in order to better integrate and manage various energy resources in urban areas, a Deep Q-Leaning Network-Quasi Upper Confidence Bound model is innovatively constructed using deep reinforcement learning technology to learn the state and behavior mapping relationship of energy system. Use deep learning to fit complex nonlinear models to optimize the entire energy system. Compare and verify the experiment with the real energy system. The improved Deep reinforcement learning algorithm is compared with Q-learning model, PDWoLF PHC algorithm model, Quasi Upper Confidence Bound algorithm model and deep Q-Leaning Network algorithm model. The results show that the research algorithm has the smallest instantaneous error value and absolute value of frequency deviation for area control, and the average value of the research algorithm in the absolute value of the frequency deviation is reduced by 45%–73% compared to other algorithms; over time, the unit output power of the research algorithm is able to flexibly track the stochastic square wave loads. Therefore, the proposed system strategies can provide feasible solutions to meet the challenges of extreme events and promote the sustainable development and safe operation of urban energy systems.

Deep Reinforcement Learning for Optimizing Restricted Access Window in IEEE 802.11ah MAC Layer

The IEEE 802.11ah standard is introduced to address the growing scale of internet of things (IoT) applications. To reduce contention and enhance energy efficiency in the system, the restricted access window (RAW) mechanism is introduced in the medium access control (MAC) layer to manage the significant number of stations accessing the network. However, to achieve optimized network performance, it is necessary to appropriately determine the RAW parameters, including the number of RAW groups, the number of slots in each RAW, and the duration of each slot. In this paper, we optimize the configuration of RAW parameters in the uplink IEEE 802.11ah-based IoT network. To improve network throughput, we analyze and establish a RAW parameters optimization problem. To effectively cope with the complex and dynamic network conditions, we propose a deep reinforcement learning (DRL) approach to determine the preferable RAW parameters to optimize network throughput. To enhance learning efficiency and stability, we employ the proximal policy optimization (PPO) algorithm. We construct network environments with periodic and random traffic in an NS-3 simulator to validate the performance of the proposed PPO-based RAW parameters optimization algorithm. The simulation results reveal that using the PPO-based DRL algorithm, optimized RAW parameters can be obtained under different network conditions, and network throughput can be improved significantly.

Energy-saving design of underground metro vertical alignment with deep reinforcement learning

The design of metro vertical alignment greatly affects train traction energy consumption. This study develops a deep reinforcement learning (DRL) algorithm for an energy-saving design of underground metro vertical alignment and analyzes the influence of train traction energy consumption calculation factors (CFs) on energy-saving design. First, a metro train operation simulation model is adopted using the approximate integral method, and the value range of CFs as input parameters is determined. Second, the 'agent', 'state', 'action', and 'reward' of the metro vertical alignment environment and the corresponding alignment constraints are determined. Third, a Dueling Double Deep Q Network (D3QN) is constructed to design the energy-saving metro vertical alignment. The proposed model is applied to a real-world metro case study, and the influence of CFs (design speed, limiting gradient in reverse slopes, and passenger capacity) on energy-saving design is analyzed. Finally, compared to manual design, the D3QN-based design outperforms GA-based design and DQN-based design, and demonstrates a remarkable energy savings of 6.17%.

Reinforcement Learning for Closed-loop Regulation of Cardiovascular System with Vagus Nerve Stimulation: A Computational Study.

(Objective). Vagus nerve stimulation (VNS) is being investigated as a potential therapy for cardiovascular diseases including heart failure, cardiac arrhythmia, and hypertension. The lack of a systematic approach for controlling and tuning the VNS parameters poses a significant challenge. Closed-loop VNS strategies combined with artificial intelligence (AI) approaches offer a framework for systematically learning and adapting the optimal stimulation parameters. In this study, we presented an interactive AI framework using reinforcement learning (RL) for automated data-driven design of closed-loop VNS control systems in a computational study. (Approach). Multiple simulation environments with a standard application programming interface were developed to facilitate the design and evaluation of the automated data-driven closed-loop VNS control systems. These environments simulate the hemodynamic response to multi-location VNS using biophysics-based computational models of healthy and hypertensive rat cardiovascular systems in resting and exercise states. We designed and implemented the RL-based closed-loop VNS control frameworks in the context of controlling the heart rate (HR) and the mean arterial pressure (MAP) for a set point tracking task. Our experimental design included two approaches; a general policy using deep RL algorithms and a sample-efficient adaptive policy using probabilistic inference for learning and control (PILCO). (Main results). Our simulation results demonstrated the capabilities of the closed-loop RL-based approaches to learn optimal VNS control policies and to adapt to variations in the target set points and the underlying dynamics of the cardiovascular system. Our findings highlighted the trade-off between sample-efficiency and generalizability, providing insights for proper algorithm selection. Finally, we demonstrated that transfer learning improves the sample efficiency of Deep RL algorithms allowing the development of more efficient and personalized closed-loop VNS systems. (Significance). We demonstrated the capability of RL-based closed-loop VNS systems. Our approach provided a systematic adaptable framework for learning control strategies without requiring prior knowledge about the underlying dynamics.

A DRL-based Partial Charging Algorithm for Wireless Rechargeable Sensor Networks

Breakthroughs in Wireless Energy Transfer (WET) technologies have revitalized Wireless Rechargeable Sensor Networks (WRSNs). However, how to schedule mobile chargers rationally has been quite a tricky problem. Most of the current work does not consider the variability of scenarios and how many mobile chargers should be scheduled as the most appropriate for each dispatch. At the same time, the focus of most work on the mobile charger scheduling problem has always been on reducing the number of dead nodes, and the most critical metric of network performance, packet arrival rate, is relatively neglected. In this paper, we develop a DRL-based Partial Charging (DPC) algorithm. Based on the number and urgency of charging requests, we classify charging requests into four scenarios. And for each scenario, we design a corresponding request allocation algorithm. Then, a Deep Reinforcement Learning (DRL) algorithm is employed to train a decision model using environmental information to select which request allocation algorithm is optimal for the current scenario. After the allocation of charging requests is confirmed, to improve the Quality of Service (QoS), i.e., the packet arrival rate of the entire network, a partial charging scheduling algorithm is designed to maximize the total charging duration of nodes in the ideal state while ensuring that all charging requests are completed. In addition, we analyze the traffic information of the nodes and use the Analytic Hierarchy Process (AHP) to determine the importance of the nodes to compensate for the inaccurate estimation of the node’s remaining lifetime in realistic scenarios. Simulation results show that our proposed algorithm outperforms the existing algorithms regarding the number of alive nodes and packet arrival rate.

Quantum-inspired deep reinforcement learning for adaptive frequency control of low carbon park island microgrid considering renewable energy sources

The low carbon park islanded microgrid faces operational challenges due to the high variability and uncertainty of distributed renewable energy sources. These sources cause severe random disturbances that impair the frequency control performance and increase the regulation cost of the islanded microgrid, jeopardizing its safety and stability. This paper presents a data-driven intelligent load frequency control (DDI-LFC) method to address this problem. The method replaces the conventional LFC controller with an intelligent agent based on a deep reinforcement learning algorithm. To adapt to the complex islanded microgrid environment and achieve adaptive multi-objective optimal frequency control, this paper proposes the quantum-inspired maximum entropy actor-critic (QIS-MEAC) algorithm, which incorporates the quantum-inspired principle and the maximum entropy exploration strategy into the actor-critic algorithm. The algorithm transforms the experience into a quantum state and leverages the quantum features to improve the deep reinforcement learning’s experience replay mechanism, enhancing the data efficiency and robustness of the algorithm and thus the quality of DDI-LFC. The validation on the Yongxing Island isolated microgrid model of China Southern Grid (CSG) demonstrates that the proposed method utilizes the frequency regulation potential of distributed generation, and reduces the frequency deviation and generation cost.

Deep reinforcement learning for artificial upwelling energy management

The potential of artificial upwelling in stimulating seaweed growth, consequently enhancing ocean carbon sequestration has been gaining increasing attention in recent years. This has led to the development of the first solar-powered and air-lifted artificial upwelling system (AUS) in China. However, effective scheduling of the air injection system and energy storage system in dynamic marine environments remains a crucial challenge in the operation of the AUS, as it holds the potential to significantly improve system performance. To tackle this challenge, we propose a novel energy management approach that utilizes the deep reinforcement learning (DRL) algorithm to determine the optimal operational parameters of the AUS at each time interval. Specifically, we formulate the energy optimization problem as a Markov decision process and integrate the quantile network in distributional reinforcement learning with the deep dueling network to solve the problem. Through extensive simulations, we evaluate the performance of our algorithm and demonstrate its superior effectiveness over traditional rule-based approaches and other DRL algorithms in enhancing energy utilization while ensuring the secure and reliable operation of the AUS. Our findings suggest that a DRL-based approach offers a promising way to provide valuable guides for the operation of the AUS and enhance the sustainability of seaweed cultivation and carbon sequestration in the ocean.

A deep reinforcement learning algorithm framework for solving multi-objective traveling salesman problem based on feature transformation

As a special type of multi-objective combinatorial optimization problems (MOCOPs), the multi-objective traveling salesman problem (MOTSP) plays an important role in practical fields such as transportation and robot control. However, due to the complexity of its solution space and the conflicts between different objectives, it is difficult to obtain satisfactory solutions in a short time. This paper proposes an end-to-end algorithm framework for solving MOTSP based on deep reinforcement learning (DRL). By decomposing strategies, solving MOTSP is transformed into solving multiple single-objective optimization subproblems. Through linear transformation, the features of the MOTSP are combined with the weights of the objective function. Subsequently, a modified graph pointer network (GPN) model is used to solve the decomposed subproblems. Compared with the previous DRL model, the proposed algorithm can solve all the subproblems using only one model without adding weight information as input features. Furthermore, our algorithm can output a corresponding solution for each weight, which increases the diversity of solutions. In order to verify the performance of our proposed algorithm, it is compared with four classical evolutionary algorithms and two DRL algorithms on several MOTSP instances. The comparison shows that our proposed algorithm outperforms the compared algorithms both in terms of training time and the quality of the resulting solutions.

Multi-agent deep reinforcement learning for trajectory planning in UAVs-assisted mobile edge computing with heterogeneous requirements

In heterogeneous wireless networks, massive user equipments (UEs) generate computing tasks with time-varying heterogeneous requirements. To improve the service quality, this paper formulates a unmanned aerial vehicles (UAVs)-assisted mobile edge computing (MEC) framework for time-varying heterogeneous task requirements. In the framework, the task delay and the number of successfully executed tasks are optimized by jointly controlling the trajectories of multiple UAVs. To address the considered trajectory planning optimization problem, a collaborative multi-agent deep reinforcement learning (MADRL) algorithm is proposed, where each UAV is regarded as a learning agent. First, a counterfactual inference based personalized policy update mechanism is proposed to evaluate the independent policy of agents by comparing the policy with a designed counterfactual policy. Based on this idea, each agent updates a personalized policy from both group and individual interests to improve its cooperation ability in dynamic and complex environments. Then, a diversified experience sampling mechanism is proposed to enhance the efficiency of policy evaluation and update with rich experiences provided by the environment interaction and the modified whale optimization algorithm. Finally, evaluation results demonstrate the superiority and effectiveness of the proposed MADRL algorithm.

Impact of memory on beamforming optimization for DDPG-assisted RIS-multi-user system

Recently, reconfigurable intelligent surfaces (RIS) have garnered considerable attention as indispensable components driving the evolution of future 6G wireless communication systems. This heightened interest is primarily attributed to notable advancements in programmable meta-material fabrication, which enable the creation of highly versatile and adaptable surfaces. These surfaces, often referred to as intelligent reflecting arrays, represent a significant departure from the conventional capabilities of massive multiple-input multiple-output (MIMO) systems, thereby catalyzing the emergence of intelligent radio environments characterized by enhanced flexibility and efficiency. Our study is dedicated to the comprehensive exploration of coordinated design strategies that encompass both the transmission beamforming matrix at the base station and the phase shift matrix at the RIS. Leveraging recent breakthroughs in deep reinforcement learning (DRL), our approach harnesses the power of a Long Short-Term Memory (LSTM) based DRL algorithm. This algorithm orchestrates the joint design process through iterative interactions with the environment, strategically guided by predefined rewards operating within a continuous framework of states and actions. While recent research has underscored the efficacy of LSTM-based architectures in bolstering the learning capacity of reinforcement learning (RL) algorithms and simplifying the search process, our investigation reveals a nuanced insight. Contrary to prevailing trends, we find that the integration of memory into the Deep Deterministic Policy Gradient (DDPG) with LSTM (DDPG-LSTM) algorithm yields unexpected consequences, negatively impacting the system's overall performance. Through preliminary simulation results, we empirically demonstrate the adverse effect of memory on DDPG performance when applied to RIS systems. This finding not only sheds light on the complexities of optimizing 6G wireless communication systems but also underscores the importance of careful algorithmic design and parameter tuning in achieving desired outcomes in emerging technologies.

An advanced deep reinforcement learning algorithm for three-layer D2D-edge-cloud computing architecture for efficient task offloading in the Internet of Things

The Internet of Things (IoTs) has transformed the digital landscape by interconnecting billions of devices worldwide, paving the way for smart cities, homes, and industries. With the exponential growth of IoT devices and the vast amount of data they generate, concerns have arisen regarding efficient task-offloading strategies. Traditional cloud and edge computing methods, paired with basic Machine Learning (ML) algorithms, face several challenges in this regard. In this paper, we propose a novel approach to task offloading in a Device-to-Device (D2D)-Edge-Cloud computing using the Rainbow Deep Q-Network (DQN), an advanced Deep Reinforcement Learning (DRL) algorithm. This algorithm utilizes advanced neural networks to optimize task offloading in the three-tier framework. It balances the trade-offs among D2D, Device-to-Edge (D2E), and Device/Edge-to-Cloud (D2C/E2C) communications, benefiting both end users and servers. These networks leverage Deep Learning (DL) to discern patterns, evaluate potential offloading decisions, and adapt in real time to dynamic environments. We compared our proposed algorithm against other state-of-the-art methods. Through rigorous simulations, we achieved remarkable improvements across key metrics: an increase in energy efficiency by 29.8%, a 27.5% reduction in latency, and a 43.1% surge in utility.

Rapid Adaptation for Active Pantograph Control in High-Speed Railway via Deep Meta Reinforcement Learning.

Active pantograph control is the most promising technique for reducing contact force (CF) fluctuation and improving the train's current collection quality. Existing solutions, however, suffer from two significant limitations: 1) they are incapable of dealing with the various pantograph types, catenary line operating conditions, changing operating speeds, and contingencies well and 2) it is challenging to implement in practical systems due to the lack of rapid adaptability to a new pantograph-catenary system (PCS) operating conditions and environmental disturbances. In this work, we alleviate these problems by developing a revolutionary context-based deep meta-reinforcement learning (CB-DMRL) algorithm. The proposed CB-DMRL algorithm combines Bayesian optimization (BO) with deep reinforcement learning (DRL), allowing the general agent to adapt to new tasks quickly and efficiently. We evaluated the CB-DMRL algorithm's performance on a proven PCS model. The experimental results demonstrate that meta-training DRL policies with latent space swiftly adapt to new operating conditions and unknown perturbations. The meta-agent adapts quickly after two iterations with a high reward, which require only ten spans, approximately equal to 0.5 km of PCS interaction data. Compared with state-of-the-art DRL algorithms and traditional solutions, the proposed method can promptly traverse scenario changes and reduce CF fluctuations, resulting in an excellent performance.

A Deep Recurrent Reinforcement Learning Approach for Enhanced MPPT in PV Systems

Under partial shading condition (PSC), for efficient usage, solar photovoltaic (PV) systems are required to operate at global maximum power point (GMPP). A maximum power point tracking (MPPT) controller is usually employed for this task. In this work, for MPPT under PSC, a recurrent deep reinforcement learning (DRL) approach has been developed and investigated. The optimized neural network, augmented by long short-term memory (LSTM), is trained using the proximal policy optimization (PPO) DRL algorithm. The MPPT performance of the proposed scheme was assessed using the GMPP attainment accuracy (%), on several realistic static and dynamic test cases. The presented investigations revealed the proposed controller's average MPPT accuracy of 97.79% for the considered random static PSCs. Similarly, average MPPT accuracies of 95.69% under changing irradiance, 95.70% under change in temperature and 95.77% under changes in both irradiance and temperature were recorded. These accuracies are found to be significantly superior to recent reported works. The resulting performance enhancement is attributed to the inclusion of LSTM, which allowed the agent to retain information about past states and actions, enabling it to make informed decisions. Therefore, based on the presented investigations, it is concluded that the proposed PPO-LSTM approach is an excellent MPPT alternative for PV systems.

Towards efficient RAN slicing: A deep hierarchical reinforcement learning approach

Radio access network (RAN) slicing can significantly improve network flexibility and resource utilization efficiency. Generally, deep reinforcement learning (DRL) is a prevailing approach to implementing practicable resource management in RAN slicing. However, this demand-aware resource allocation suffers from long training time and slow execution efficiency. This can lead to the inability to quickly achieve the desired spectral efficiency and service level agreement satisfaction rate. To tackle this issue, we propose a novel RAN slicing resource allocation framework based on a deep hierarchical RL framework for efficient resource scheduling. Structurally, our framework consists of a policy selection network and a policy evaluation network. In particular, a newly built action acceleration unit can achieve quick reward accumulation, thus speeding up the optimal policy search. Extensive simulations show that our proposal has higher system utility and faster convergence compared to the state-of-the-art DRL algorithms.

Deep deterministic policy gradient algorithm: A systematic review

Deep Reinforcement Learning (DRL) has gained significant adoption in diverse fields and applications, mainly due to its proficiency in resolving complicated decision-making problems in spaces with high-dimensional states and actions. Deep Deterministic Policy Gradient (DDPG) is a well-known DRL algorithm that adopts an actor-critic approach, synthesizing the advantages of value-based and policy-based reinforcement learning methods. The aim of this study is to provide a thorough examination of the latest developments, patterns, obstacles, and potential opportunities related to DDPG. A systematic search was conducted using relevant academic databases (Scopus, Web of Science, and ScienceDirect) to identify 85 relevant studies published in the last five years (2018-2023). We provide a comprehensive overview of the key concepts and components of DDPG, including its formulation, implementation, and training. Then, we highlight the various applications and domains of DDPG, including Autonomous Driving, Unmanned Aerial Vehicles, Resource Allocation, Communications and the Internet of Things, Robotics, and Finance. Additionally, we provide an in-depth comparison of DDPG with other DRL algorithms and traditional RL methods, highlighting its strengths and weaknesses. We believe that this review will be an essential resource for researchers, offering them valuable insights into the methods and techniques utilized in the field of DRL and DDPG.

Stochastic Integrated Actor-Critic for Deep Reinforcement Learning.

We propose a deep stochastic actor-critic algorithm with an integrated network architecture and fewer parameters. We address stabilization of the learning procedure via an adaptive objective to the critic's loss and a smaller learning rate for the shared parameters between the actor and the critic. Moreover, we propose a mixed on-off policy exploration strategy to speed up learning. Experiments illustrate that our algorithm reduces the sample complexity by 50%-93% compared with the state-of-the-art deep reinforcement learning (RL) algorithms twin delayed deep deterministic policy gradient (TD3), soft actor-critic (SAC), proximal policy optimization (PPO), advantage actor-critic (A2C), and interpolated policy gradient (IPG) over continuous control tasks LunarLander, BipedalWalker, BipedalWalkerHardCore, Ant, and Minitaur in the OpenAI Gym.

Robust and adaptive deep reinforcement learning for enhancing flow control around a square cylinder with varying Reynolds numbers

The present study applies a Deep Reinforcement Learning (DRL) algorithm to Active Flow Control (AFC) of a two-dimensional flow around a confined square cylinder. Specifically, the Soft Actor-Critic (SAC) algorithm is employed to modulate the flow of a pair of synthetic jets placed on the upper and lower surfaces of the confined squared cylinder in flow configurations characterized by Re of 100, 200, 300, and 400. The investigation starts with an analysis of the baseline flow in the absence of active control. It is observed that at Re = 100 and Re = 200, the vortex shedding exhibits mono-frequency characteristics. Conversely, at Re = 300 and Re = 400, the vortex shedding is dominated by multiple frequencies, which is indicative of more complex flow features. With the application of the SAC algorithm, we demonstrate the capability of DRL-based control in effectively suppressing vortex shedding, while significantly diminishing drag and fluctuations in lift. Quantitatively, the data-driven active control strategy results in a drag reduction of approximately 14.4%, 26.4%, 38.9%, and 47.0% for Re = 100, 200, 300, and 400, respectively. To understand the underlying control mechanism, we also present detailed flow field comparisons, which showcase the adaptability of DRL in devising distinct control strategies tailored to the dynamic conditions at varying Re. These findings substantiate the ability of DRL to control chaotic, multi-frequency dominated vortex shedding phenomena, underscoring the robustness of DRL in complex AFC problems.

Global optimization strategy of prosumer data center system operation based on multi-agent deep reinforcement learning

The escalating issues of high energy consumption and carbon emissions in data centers (DCs) necessitate the optimization of system operations. However, early optimization strategies were overly simplistic and lacked automated updating and iterative capabilities. With the evolution of artificial intelligence (AI), researchers have applied deep reinforcement learning (DRL) algorithms to system operations. However, the optimization focus has been limited to the internal systems, lacking global optimization. In this paper, a global optimization control strategy based on the Dueling double-deep Q network (D3QN) and value decomposition network (VDN) algorithms is proposed to make the DCs system operate more closely with the upstream, midstream, and downstream. By adjusting battery charging/discharging capacity, computational workload, and waste heat utilization heating temperature global synergistic optimization is achieved. Compared with without optimization, renewable energy waste, operation cost, total electricity consumption, and grid electricity consumption are reduced by 18.37%, 9.78%, 4.01%, and 29.74%, respectively. Additionally, a detailed comparison between non-algorithmic optimization and algorithmic optimization is provided, offering valuable insights for substantial energy savings and emissions reduction in DCs. The results demonstrate the importance of fully exploring the interactive potential between upstream energy supply, midstream computational workload, and downstream waste heat recovery to achieve synergistic global optimization of "computing power", "thermal energy" and "electrical energy" for the sustainable and green development of DCs or other prosumer buildings.

Multi-agent reinforcement learning based computation offloading and resource allocation for LEO Satellite edge computing networks

Due to the limitations caused by geographical conditions and economic requirements, it is difficult to provide computing services by terrestrial networks for mobile terminals in remote areas. To address this issue, mobile edge computing (MEC) servers can be deployed in the low earth orbit (LEO) satellites to act as a complement and accommodate the unserved terminals. However, offloading computing tasks to servers in satellites may increase the energy consumption of ground terminals. Considering the limited battery capacity of ground terminals, how to perform the computation offloading and resource allocation are key challenges in the LEO satellite edge computing networks. Therefore, in this paper, we investigate the energy minimization problem for LEO satellite edge computing networks, where a multi-agent deep reinforcement learning algorithm with global rewards is proposed to optimize the transmit power, CPU frequency, bit allocation, offloading decision and bandwidth allocation via a decentralized method. Simulation results show that our proposed algorithm can converge faster. Most importantly, compared with the random algorithm, the proximal policy optimization (PPO) algorithm, and the deep deterministic policy gradient (DDPG) algorithm, the ground terminals’ energy consumption can be effectively reduced by our proposed algorithm.

Online energy management strategy for ammonia-hydrogen hybrid electric vehicles harnessing deep reinforcement learning

Powertrain electrification and fuel decarbonization play pivotal roles in the pursuit of carbon peak and carbon neutrality within the realm of transportation. Ammonia and hydrogen are essential carbon-neutral fuels suited for deployment in heavy-duty, long-haul transportation applications. The technological complementarity of ammonia and hydrogen can be realized in the current context by combining various energy conversion devices. Energy management strategy (EMS) is one of the critical technologies in hybrid systems. In this study, a novel EMS based on deep reinforcement learning (DRL) is proposed for a heavy-duty automotive hybrid system containing an ammonia-hydrogen internal combustion engine (ICE), a fuel cell system (FCS), an ammonia electrolysis cell (AEC), and Li-ion batteries (LB). To address the protracted training times and convergence challenges intrinsic to DRL, a fuzzy logic control (FLC) is introduced to provide expert demonstrations during the DRL agent's learning process. The incorporation of FLC not only facilitates the convergence speed of the DRL algorithm, but also strikes a good balance between the operation efficiency improvement and battery SOC maintenance. The simulation results show that the proposed EMS improves average efficiency by 2% with a modest 0.75% SOC reduction across four operational scenarios compared to the FLC-based EMS. After thorough validation of the proposed EMS, a parametric study is conducted to examine three key parameters: charge mode threshold, FCS power change rate limit, and hybrid mode threshold. The study aims to offer valuable insights for formulating the EMS of the ammonia-hydrogen hybrid system.