Energy Management Of Hybrid Electric Vehicles Research Articles

Plug‐In Hybrid Electric Vehicle Energy Management with Clutch Engagement Control via Continuous‐Discrete Reinforcement Learning

Energy management strategy (EMS) is a key technology for plug‐in hybrid electric vehicles (PHEVs). The energy management of certain series–parallel PHEVs involves the control of continuous variables, such as engine torque, and discrete variables, such as clutch engagement/disengagement. Herein, a control‐oriented model is established for a series–parallel plug‐in hybrid system with clutch engagement control from the perspective of mixed‐integer programming. Subsequently, an EMS based on continuous‐discrete reinforcement learning (CDRL), which enables simultaneous output of continuous and discrete variables, is designed. During training, state‐of‐charge (SOC) randomization is introduced to ensure that the hybrid system exhibits optimal energy‐saving performance in both high and low SOC. Finally, the effectiveness of the proposed CDRL strategy is verified by comparing EMS based on charge‐depleting charge‐sustaining (CD‐CS) with rule‐based clutch engagement control and dynamic programming (DP). In the simulation results, it is shown that, under a high SOC, the CDRL strategy proposed in this article can improve energy efficiency by 8.3% compared to CD‐CS, and the energy consumption is just 6.6% higher than the global optimum based on DP, while under a low SOC, the numbers are 4.1% and 3.9%, respectively.

Energy Management for Hybrid Electric Vehicles Using Safe Hybrid-Action Reinforcement Learning

Reinforcement learning has shown success in solving complex control problems, yet safety remains paramount in engineering applications like energy management systems (EMS), particularly in hybrid electric vehicles (HEVs). An effective EMS is crucial for coordinating power flow while ensuring safety, such as maintaining the battery state of charge within safe limits, which presents a challenging task. Traditional reinforcement learning struggles with safety constraints, and the penalty method often leads to suboptimal performance. This study introduces Lagrangian-based parameterized soft actor–critic (PASACLag), a novel safe hybrid-action reinforcement learning algorithm for HEV energy management. PASACLag utilizes a unique composite action representation to handle continuous actions (e.g., engine torque) and discrete actions (e.g., gear shift and clutch engagement) concurrently. It integrates a Lagrangian method to separately address control objectives and constraints, simplifying the reward function and enhancing safety. We evaluate PASACLag’s performance using the World Harmonized Vehicle Cycle (901 s), with a generalization analysis of four different cycles. The results indicate that PASACLag achieves a less than 10% increase in fuel consumption compared to dynamic programming. Moreover, PASACLag surpasses PASAC, an unsafe counterpart using penalty methods, in fuel economy and constraint satisfaction metrics during generalization. These findings highlight PASACLag’s effectiveness in acquiring complex EMS for control within a hybrid action space while prioritizing safety.

OPTIMISATION OF HYBRID ELECTRIC VEHICLE ENERGY MANAGEMENT USING MACHINE LEARNING

The focus of the research is on the optimization of the hybrid electric vehicle energy management using machine learning. The objective of this research is to develop the algorithm by using Support Vector Machine (SVM) and to identify the optimal operation mode for SHEV based on the power demand use by using the algorithm. First of all, this research focus on the series hybrid electric vehicle and the vehicle used is an all-terrain vehicle (ATV). Suitable formula will be used to complete the modeling of the vehicle in Energetic Macroscopic Representation (EMR) form and the model constructed by using Matlab Simulink. Then, Support Vector Machine is used to optimize the energy management in the vehicle. The training data from New European Driving Cycle (NEDC) and Worldwide harmonized Light vehicles Test Cycles (WLTC) with 3 classes will put be inside the SVM to undergo training and then the optimal operation modes for each driving cycle will be obtained by using Linear SVM. Then, the obtained results which are the predicted operation mode using the driving cycles and are plotted in the graph. The pattern of the graph is analyzed and then the best predicted operation mode with the highest accuracy among all the driving cycles is chosen. The trained model of the driving cycle with the highest accuracy is used to predict the optimal operation mode for ATV so that to have higher efficiency in energy management by using the Classification Learner inside the Matlab.

FlexNet: A warm start method for deep reinforcement learning in hybrid electric vehicle energy management applications

Deep reinforcement learning (DRL) has been widely studied in the energy management of hybrid electric vehicles (HEVs) for its remarkable energy efficiency improvement compared to conventional methods. However, how to alleviate the time consumption of training a stable reinforcement learning agent still needs to be solved in real-world implementation. This study presents a human expert knowledge encoded ‘warm start’ method with the flexibility to change the neural network architecture. The expert knowledge is encoded in a decision tree which then initializes the weights and bias of the DRL neural network. Compared with another fixed architecture warm start method, the proposed FlexNet exhibits improved learning speed by 60.8 % and 88.8 % in action space 50 and 100, respectively. The energy consumption by the proposed FlexNet EMS method is 12.2 % and 6.4 % better than rule-based and equivalent consumption minimization strategy, respectively. This proposed warm start method can reduce learning time and increase energy efficiency in various energy management applications.

Safe Deep Reinforcement Learning-Based Constrained Optimal Control Scheme for HEV Energy Management

Considering physical constraints in online optimization and training safety is a challenge for implementation of deep reinforcement learning (DRL) algorithm. Especially for the non-linear system, the mapping relationship between output action of the agent and the control signals is difficult to obtain. This paper proposes a novel DRL framework for online optimization in energy management of a power-split hybrid electric vehicle (HEV) which combines a neural-network (NN)-based multi-constraints optimal strategy and a rule-based-restraints system (RBRS). The proposed method named Reward Directed Policy Optimization (RDPO) adopts exterior point method (EPM) and curriculum learning (CL) to direct the agent to recognize and avoid irrational control signals and optimize the fuel economy. The EMS considering fuel consumption minimization and irrational control signals avoidance is optimized by training the agent through WLTC. A competitive fuel economy, 4.495L/100km, is achieved with no irrational control signals. Based on the online adaptability evaluation conducted, the fuel consumption of the vehicle under NEDC and CTUDC has been reduced to 4.113L/100km and 3.221L/100km, respectively, with no irrational control signals. The superiority in optimization, calculation efficiency and safety is verified through the comparisons with various DRL agents.

Automated Expert Knowledge-Based Deep Reinforcement Learning Warm Start via Decision Tree for Hybrid Electric Vehicle Energy Management

<div>Deep reinforcement learning has been utilized in different areas with significant progress, such as robotics, games, and autonomous vehicles. However, the optimal result from deep reinforcement learning is based on multiple sufficient training processes, which are time-consuming and hard to be applied in real-time vehicle energy management. This study aims to use expert knowledge to warm start the deep reinforcement learning for the energy management of a hybrid electric vehicle, thus reducing the learning time. In this study, expert domain knowledge is directly encoded to a set of rules, which can be represented by a decision tree. The agent can quickly start learning effective policies after initialization by directly transferring the logical rules from the decision tree into neural network weights and biases. The results show that the expert knowledge-based warm start agent has a higher initial learning reward in the training process than the cold start. With more expert knowledge, the warm start shows improved performance in the initial learning stage compared to the warm start method with less expert knowledge. The results indicate that the proposed warm start method requires 76.7% less time to achieve convergence than the cold start. The proposed warm start method is also compared with the conventional rule-based method and equivalent consumption minimization strategy. The proposed warm start method reduces energy consumption by 8.62% and 3.62% compared with the two baseline methods, respectively. The results of this work can facilitate the expert knowledge-based deep reinforcement learning warm start in hybrid electric vehicle energy management problems.</div>

Cooperative optimization of speed planning and energy management for hybrid electric vehicles based on Nash equilibrium

The longitudinal automatic driving of hybrid electric vehicles (HEVs) is a complex multi-objective problem, and the coupling relationship between speed planning and energy management is difficult to reveal using the commonly used hierarchical control algorithm. Thus, in this paper, a collaborative control strategy is proposed for cruise speed optimization and energy management of HEVs based on Nash equilibrium. The strategy in this paper can optimize the control of vehicle-level and hybrid-system-level at the same time. First, this paper builds the key component model of HEV under the Simulink environment, which lays the foundation for vehicle control. Then, a multi-objective problem-solving framework based on game theory is proposed aiming at the economy and safety of HEV cruising. Afterward, a weight factor adjustment method is proposed based on fuzzy logic to judge the brake/drive logic. Finally, the utility functions are used for coordinated control based on Nash equilibrium. The simulation results show that the control method proposed in this paper has improved in various aspects as compared with hierarchical model predictive control, including vehicle following, safety, and comfort. In addition, this strategy can significantly reduce energy consumption costs while meeting the cruise speed requirements of vehicles. In the whole driving cycle, the strategy proposed in this paper has a high termination of SOC and a fuel consumption reduction of 15.33%. It can better deal with the relationship between speed planning and torque distribution than the traditional hierarchical control method.

Recent Advances and Applications of AI-Based Mathematical Modeling in Predictive Control of Hybrid Electric Vehicle Energy Management in China

Artificial intelligence is widely used in mathematical modeling. The technical means in mathematical modeling are more and more diversified, especially the application of artificial intelligence algorithm greatly promotes the development of mathematical modeling. In recent years, because of its great influence on the fuel consumption, output power and exhaust performance of automobiles, the control strategy has become a research hotspot and focus in automobile R&D industry. Therefore, based on the relevant research results in recent years, after studying and analyzing the typical control strategies of hybrid vehicles, this paper finally puts forward the energy management strategy of hybrid vehicles based on model predictive control (MPC), and strives to contribute to the academic research of energy management strategies of hybrid vehicles.

Application of a new type of lithium‑sulfur battery and reinforcement learning in plug-in hybrid electric vehicle energy management

The continuous increase in vehicle ownership has caused overall energy consumption to increase rapidly. Developing new energy vehicle technologies and improving energy utilization efficiency are significant in saving energy. Plug-in hybrid electric vehicles (PHEVs) present a practical solution to the arising energy shortage concerns. However, existing battery technologies restrict PHEV application as the most popular lithium-ion battery has a relatively high capital cost and degradation during service time. This paper studies the application of a new type of lithium‑sulfur (LiS) battery with bilateral solid electrolyte interphases in the PHEV. Compared with metals such as cobalt and nickel used in conventional lithium-ion batteries, sulfur utilized in LiS is cheaper and easier to manufacture. The high energy density of the new LiS battery also provides a longer range for PHEVs. In this paper, a PHEV propulsion system model is introduced, which includes vehicle dynamics, engine, electric motor, and LiS battery models. Dynamic programming is formulated as a benchmark energy management strategy to reduce energy consumption. Besides the offline global optimal benchmark from dynamic programming, the real-time performance of the LiS battery is evaluated by Q-learning and rule-based strategies. For a more comprehensive validation, both light-duty vehicles and heavy-duty vehicles are considered. Compared with lithium-ion batteries, the new LiS battery reduces the fuel consumption by up to 14.63 % and battery degradation by up to 82.37 %.

Discrete Mixed-Integer Shooting (DMIS): Algorithm and Application to Plug-In Hybrid Electric Vehicle Energy Management Accounting for Fuel Cranking and Actual Powertrain Efficiency Maps

This article presents an algorithm for mixed-integer nonlinear optimal control problems (OCPs) formulated directly in discrete time. In our current application, we address the energy management of a plug-in hybrid electric vehicle (PHEV) seeking to achieve minimum trip fuel consumption under a terminal state-of-charge (SOC) equality constraint while avoiding powertrain efficiency map approximations in order to maximize its fuel economy (FE). An additional engine cranking state with a nonsmooth state and control transition cost is then incorporated to prevent undesired engine on/off behaviors. We pursue a single shooting-based numerical algorithm, motivated by the success of the computationally efficient Pontryagin’s Minimum Principle (PMP)-based single shooting in achieving similar FE results as dynamic programming (DP) for the power-split optimization problem without the additional engine cranking state shown in prior work. The Discrete Mixed-Integer Shooting (DMIS) algorithm, with costate backward-in-time propagation, is presented here, avoiding unstable shooting iterations that are typically encountered in PMP-based single shooting. When applying DMIS to solve the power-split problem with cranking fuel consideration, we show through simulation that it leads to better FE results than the PMP-based single shooting without the cranking state, thus demonstrating the effectiveness of the DMIS algorithm. Finally, experimental dynamometer test results with the model predictive powertrain controller solved by DMIS for minimum trip fuel consumption are presented and discussed, demonstrating its real-time execution and the cranking fuel reduction capability.

Cooperative energy management and eco-driving of plug-in hybrid electric vehicle via multi-agent reinforcement learning

The advanced cruise control system has expanded the energy-saving potential of the hybrid electric vehicle (HEV). Despite this, most energy-saving researches for HEV either only optimize the energy management strategy (EMS) or integrate eco-driving through a hierarchically optimized assumption that optimizes EMS and eco-driving separately. Such kinds of approaches may lead to sub-optimal results. To fill this gap, we design a multi-agent reinforcement learning (MARL) based optimal energy-saving strategy for HEV, achieving a cooperative control on the powertrain and car-following behaviors to minimize the energy consumption and keep a safe following distance simultaneously. Specifically, a plug-in HEV model is regarded as the research object in this paper. Firstly, the HEV energy management problem in the car-following scenario is decomposed into a multi-agent cooperative task into two subtasks, each of which can conduct interactive learning through cooperative optimization. Secondly, the energy-saving strategy is designed, called the independent soft actor–critic, which consists of a car-following agent and an energy management agent. Finally, the performance of velocity tracking and energy-saving are validated under different driving cycles. In comparison to the state-of-the-art hierarchical model predictive control (MPC) strategy, the proposed MARL method can reduce fuel consumption by 15.8% while ensuring safety and comfort.

Closed-form solution to the dynamic programming for a heavy-duty parallel hybrid vehicle energy management

Dynamic programming (DP) is frequently used to obtain the optimal solution to the hybrid electric vehicle (HEV) energy management. The trade-off between the accuracy and the computational effort is the biggest problem for the DP method. The closed-form solution to the DP is proposed to solve this problem. Firstly, the affine linear model of the engine fuel rate is obtained based on engine test data. The piecewise linear approximation of the motor power demand is obtained considering the different energy flows in the charging and discharging stages of the battery. Then, the second-order Taylor expansion for the cost matrix at each time and state grid point is introduced to get the closed-form solution of the optimal torque split. The results show that this method can greatly reduce the computing burden by 93% while ensuring near-optimal fuel economy compared with the conventional DP method.

Practical application of energy management strategy for hybrid electric vehicles based on intelligent and connected technologies: Development stages, challenges, and future trends

The rapid development of intelligent and connected technologies is conducive to the efficient energy utilization of hybrid electric vehicles (HEVs). However, most energy management strategies (EMSs) with optimized, intelligent, and connected functions have not been directly applied to such vehicles because existing technical conditions cannot meet the theoretical requirements of complex EMSs. Therefore, based on the mapping relationship between the information decision-making ability and the energy management effect, this study is the first to propose four development stages of HEV energy management practical application as follows: energy management based on instantaneous driving cycles (Stage 1 or S1); energy management based on forward driving cycle prediction (Stage 2 or S2); energy management based on global driving cycle prediction (Stage 3 or S3); and energy management based on autonomous velocity planning (Stage 4 or S4). The key technologies of each development stage are not independent, i.e., they complement each other in the process of practical application development. Furthermore, realizing energy management practical applications not only requires novel algorithm models but also involves several challenges such as acquiring and processing multi-source information, predicting the vehicle power demand in the spatial–temporal domain during travel, and determining the vehicle control characteristics and ability. Finally, according to the development goals of energy management, this study proposes an implementation framework for HEV energy management in higher development stages, namely cooperative vehicle–edge–cloud for intelligent energy management, i.e., CVEC- IEM, which executes information decision tasks on different computing platforms and realizes interconnection and interaction to provide development directions and goals for the efficient utilization of energy and successful deployment of HEV practical applications.

An Adaptive Hierarchical Energy Management Strategy for Hybrid Electric Vehicles Combining Heuristic Domain Knowledge and Data-Driven Deep Reinforcement Learning

With the development of artificial intelligence, there has been a growing interest in machine learning-based control strategy, among which reinforcement learning (RL) has opened up a new direction in the field of hybrid electric vehicle (HEV) energy management. However, the issues of the current RL setting ranging from inappropriate battery state-of-charge (SOC) constraint to ineffective and risky exploration make it inapplicable to many industrial energy management strategy (EMS) tasks. To address this, an adaptive hierarchical EMS combining heuristic equivalent consumption minimization strategy (ECMS) knowledge and deep deterministic policy gradient (DDPG), which is a state-of-the-art data-driven RL algorithm, is proposed in this work. For comparison purposes, the proposed strategy is contrasted with dynamic programming (DP), proportion integration differentiation (PID)-based adaptive ECMS, and rule-based and standard RL-based counterparts, and the results show that the fuel consumption after SOC correction for the proposed strategy is very close to that of the DP-based control and lower than that of the other three benchmark strategies. Considering that the proposed strategy can make better use of the RL techniques while realizing an effective, efficient, and safe exploration in a data-driven manner, it may become a strong foothold for future RL-based EMS to build on, especially when the controller has to be trained directly and from scratch in a real-world environment.

Energy Management of Hybrid Electric Urban Bus by Off-Line Dynamic Programming Optimization and One-Step Look-Ahead Rollout

Due to the growing air quality concern in urban areas and rising fuel prices, urban bus fleets are progressively turning to hybrid electric vehicles (HEVs) which show higher efficiency and lower emissions in comparison with conventional vehicles. HEVs can reduce fuel consumption and emissions by combining different energy sources (i.e., fuel and batteries). In this sense, the performance of HEVs is strongly dependent on the energy management strategy (EMS) which coordinates the energy sources available to exploit their potential. While most EMSs are calibrated for general driving conditions, this paper proposes to adapt the EMS to the specific driving conditions on a particular bus route. The proposed algorithm relies on the fact that partial information on the driving cycle can be assumed since, in the case of a urban bus, the considered route is periodically covered. According to this hypothesis, the strategy presented in this paper is based on estimating the driving cycle from a previous trip of the bus in the considered route. This initial driving cycle is used to compute the theoretical optimal solution by dynamic programming. The obtained control policy (particularly the cost-to-go matrix) is stored and used in the subsequent driving cycles by applying one-step look-ahead roll out, then, adapting the EMS to the actual driving conditions but exploiting the similarities with previous cycles in the same route. To justify the proposed strategy, the paper discusses the common patterns in different driving cycles of the same bus route, pointing out several metrics that show how a single cycle captures most of the key parameters for EMS optimization. Then, the proposed algorithm (off-line dynamic programming optimization and one-step look-ahead rollout) is described. Results obtained by simulation show that the proposed method is able to keep the battery charge within the required range and achieve near-optimal performance, with only a 1.9% increase in fuel consumption with regards to the theoretical optimum. As a reference for comparison, the equivalent consumption minimization strategy (ECMS), which is the most widespread algorithm for HEV energy management, produces an increase in fuel consumption with respect to the optimal solution of 11%.

Deep Reinforcement Learning Based on Driver Experience Embedding for Energy Management Strategies in Hybrid Electric Vehicles

Reinforcement learning (RL) is a solution with great potential for hybrid electric vehicle (HEV) energy management strategies (EMS). However, traditional deep reinforcement learning (DRL) suffers from inefficiency and poor stability during random exploration in action space, so it is necessary to model some advanced driver experience knowledge and combine it with DRL. Herein, an advanced driver experience (DE) model of traffic congestion level and power matching is constructed based on fuzzy clustering and embedded into DRL. The results show that the DE embedding improves the training convergence efficiency of DRL on a power‐split HEV model, where it improves the convergence of the deep deterministic policy gradient (DDPG) by 46.2%. As DE can better adjust engine operating points and vehicle drive modes under various driving cycles, it enables DDPG to improve fuel economy by ≈6.29% while maintaining the terminal state of charge. This study aims to improve the efficiency of action space exploration and optimize the DRL learning strategy, so as to provide a theoretical basis for the design and development of EMS.

Hybrid Electric Vehicle Energy Online Allocation Strategy Based on Fuzzy Reinforcement Learning

Abstract To lessen fuel consumption of hybrid electric vehicle, energy online distribution strategy based on fuzzy reinforcement learning is proposed. Vehicle dynamic model and key component model are built, so that hybrid power system simulation model is given. Energy distribution problem of hybrid electric vehicle is transferred to constrained optimization problem of stochastic dynamic system through modelling, and solving problem based on fuzzy reinforcement learning is put forwarded. Fuzzy reinforcement learning is used to optimize fuzzy inference system real-time, which makes fuzzy inference system adjust with driving cycle adaptively, so that optimal control under any driving cycle come true. Clarified by simulation, fuel consumption under the control of rule-based is 3.89L/km, and it is 3.45L/km under the control of fuzzy reinforcement learsning, which saves 11.31% fuel consumption compared with rule-based. The data above proves validity of fuzzy reinforcement learning on hybrid electric vehicle energy management.

Generalization ability of hybrid electric vehicle energy management strategy based on reinforcement learning method

Energy management is a fundamental task of a hybrid electric vehicle. However, dealing with multiple hybrid electric vehicles would be very time consuming, and developing a separate management strategy for each model is a huge workload to. Based on the above problems, this paper investigates the generalization capability of energy management strategies for hybrid electric vehicles. To improve the generalization of energy management strategies, a multi-agent reinforcement learning algorithm is proposed. To achieve this goal, the first analysis from the state values of reinforcement learning in the state selection, if all the typical features of the vehicle operation are added to the reinforcement learning algorithm, then it will make the model have a certain generalization ability. Then, with the help of the auxiliary agent, the reward value of reinforcement learning can be improved by using KL-divergence. The training and validation results show that the strategy can also achieve the training effect when tested on new models. In addition, a new driving cycle is selected for environmental testing, and the results show that the method also has strong generalization ability.

State-of-the-art review of fuel cell hybrid electric vehicle energy management systems

<abstract> <p>The primary purpose of fuel cell hybrid electric vehicles (FCHEVs) is to tackle the challenge of environmental pollution associated with road transport. However, to benefit from the enormous advantages presented by FCHEVs, an appropriate energy management system (EMS) is necessary for effective power distribution between the fuel cell and the energy storage systems (ESSs). The past decade has brought a significant increase in the number of FCHEVs, with different EMSs having been implemented due to technology advancement and government policies. These methods are broadly categorised into rule-based EMS methods, machine learning methods and optimisation-based control methods. Therefore, this paper presents a systematic literature review on the different EMSs and strategies used in FCHEVs, with special focus on fuel cell/lithium-ion battery hybrid electric vehicles. The contribution of this study is that it presents a quantitative evaluation of the different EMSs selected by comparing and categorising them according to principles, technology maturity, advantages and disadvantages. In addition, considering the drawbacks of some EMSs, gaps were highlighted for future research to create the pathway for comprehensive emerging solutions. Therefore, the results of this paper will be beneficial to researchers and electric vehicle designers saddled with the responsibility of implementing an efficient EMS for vehicular applications.</p> </abstract>

Hybrid Model Predictive Control for Hybrid Electric Vehicle Energy Management Using an Efficient Mixed-Integer Formulation

This paper proposes to use hybrid model predictive control (HMPC) for energy management in hybrid electric vehicle (HEV) using an efficient formulation. HEV has two sources of energy - electric motor and internal combustion engine (ICE) - allowing it an additional degree of freedom to optimize the ratio between the use of two energy sources. HEV energy management is crucial to exploit its potential to reduce fuel consumption and emissions. However, it is challenging to achieve the optimal solution along with the fast dynamics of HEVs. In this paper, instead of using a nonlinear, computationally expensive dynamic model of HEV, a piecewise afne (PWA) model is used to depict its behavior, implemented with multiple linear regression. The validation of the developed model is taken with standard drive cycles. Based on the PWA model, an optimal control problem of HMPC was formulated as mixed integer linear programming (MILP). Conventionally, mixed logical dynamical (MLD) system has been used in the process control field, including early studies of HEV control. This paper applies Big-M formulation which is efficient in its problem size and tight inequality relation for integer variables. The performance of the HMPC controller was examined in a simulated environment based on MATLAB/Simulink HEVP2 application. As a result, HMPC shows superior control performance than the equivalent consumption minimization strategy (ECMS).