Entropy-augmented deep reinforcement learning with adaptive exploration for integrated energy and motor thermal management in hybrid electric vehicles

In this paper, the hybrid electric vehicle (HEV) energy management optimization method is proposed based on deep learning (DL) model predictive control. Through empirical research combined with the questionnaire survey, this article not only provides a new perspective and practical basis but also improves the efficiency and accuracy of the model by improving the relevant algorithms. The study first analyzes the importance of HEV energy management and reviews the existing literature. Then, the optimization method of HEV energy management based on the deep learning model is introduced in detail, including the composition of energy management for hybrid electric vehicles, the structure and working principle of the deep learning model, especially the backpropagation neural network (BPNN) and the convolutional neural network (CNN), and the steps of application of deep learning in energy management. In the experimental part, questionnaire data from 1,500 consumers were used to design the HEV energy management optimization scheme, and consumers’ attitudes and preferences towards HEV energy optimization were discussed. The experimental results show that the proposed model can predict HEV energy consumption under different road conditions (urban roads, highways, mountain areas, suburban areas, and construction sites), and the difference between the average predicted energy consumption and the actual energy consumption is between 0.1KWH and 0.3KWH, showing high prediction accuracy. In addition, the deep learning-based energy management strategy outperforms traditional control strategies in terms of fuel consumption (6.2 L/100 km), battery charge and discharge times (814), battery life, and CO2 emissions, significantly improving the efficiency of HEV energy. These results demonstrate the great potential and practical application value of deep learning models in the optimization of energy management of HEVs, helping to drive the development of more sustainable and efficient transportation systems.

Enabling cross-type full-knowledge transferable energy management for hybrid electric vehicles via deep transfer reinforcement learning

Road grade preview can benefit the hybrid electric vehicle (HEV) energy management because the energy efficiency performance degrades significantly when the battery state of charge (SOC) reaches its boundaries and the road grade has a great influence on the battery SOC balance. In reality the road grade in front may be a random variable as the future route may not always be known to the vehicle controller. This paper proposes a stochastic model predictive control (MPC) approach which does not require a determined route known in advance. The road grade is modeled as a Markov chain and all the possible future routes are considered in building the transition matrix. A large-time-scale HEV energy consumption model is built. The HEV energy management problem is formulated as a finite-horizon Markov decision process and solved using stochastic dynamic programming (SDP). Simulation results show that the proposed approach can prevent the battery SOC from reaching its boundaries and maintain good fuel efficiency by the stochastic road grade preview.

This paper reviews different ways to manage energy in Hybrid Electric Vehicles (HEVs) for smart cities by looking at three separate studies. Initially, it explores a structured approach to solving energy management issues in HEVs, comparing three known methods and highlighting one that can be used in real-time. Next, it discusses a creative use of Petri Nets (PNs) for managing energy, either on its own or with the Global Positioning System (GPS). This part points out the benefits of using GPS to manage energy better during different driving conditions. Lastly, the paper talks about the need to improve energy management in a specific type of HEV to address current environmental and energy challenges. It mentions the use of a Genetic Algorithm (GA) to improve energy management strategies, aiming to extend the life of the vehicle’s fuel cell and improve energy efficiency. Through these discussions, this review aims to provide a clear understanding of how energy management in HEVs can be improved in smart city settings.

The goal of this series of research is to advance hybrid electric vehicle (HEV) energy management by incorporating driver’s driving behavior and driving cycle information. To reduce HEV fuel consumption, the objectives of this research are divided into the following three parts. The first part of the research investigates the impact of driver’s behavior on the overall fuel efficiency of a hybrid electric vehicle and the energy efficiency of individual powertrain components under various driving cycles. Between the sticker number fuel economy and actual fuel economy, it is well known that a noticeable difference occur when a driver drives aggressively. To simulate aggressive driving, the input driving cycles are scaled up from the baseline driving cycles to higher levels of acceleration/deceleration. The simulation study is conducted using Autonomie®, a powertrain simulation and analysis software. The performance of the major powertrain components is analyzed when the HEV is operated at different level of aggressiveness. In the second part of the study, the vehicle driving cycles affect the performance of a hybrid vehicle control strategy and the corresponding overall performance of the vehicle. By identifying the driving cycles of a vehicle, the HEV supervisor controller system will be dynamically adapt the control strategy to the changes of vehicle driving patterns. With pattern recognition method, a driving cycle is represented by feature vectors that are formed by a set of parameters to which the driving cycle is sensitive. To establish reference driving cycle database, the representative feature vectors of four federal driving cycles are generated using feature extraction method. The performance of the presented adaptive control strategy based on driving pattern recognition is evaluated using Autonomie. In the last part of the study, a predictive control method is developed and investigated for hybrid electric vehicle energy management in effort to improve HEV fuel economy. Model Predictive Control (MPC), a predictive control method, is applied to improve the fuel economy of a power-split HEV. The study compares the performance of MPC method and conventional rule-base control method. A parametric study is conducted to understand the influence of 3 weighting factors in MPC formulation on the performance of the vehicles.

Reinforcement learning (RL) is a solution with great potential for hybrid electric vehicle (HEV) energy management strategies (EMS). However, traditional deep reinforcement learning (DRL) suffers from inefficiency and poor stability during random exploration in action space, so it is necessary to model some advanced driver experience knowledge and combine it with DRL. Herein, an advanced driver experience (DE) model of traffic congestion level and power matching is constructed based on fuzzy clustering and embedded into DRL. The results show that the DE embedding improves the training convergence efficiency of DRL on a power‐split HEV model, where it improves the convergence of the deep deterministic policy gradient (DDPG) by 46.2%. As DE can better adjust engine operating points and vehicle drive modes under various driving cycles, it enables DDPG to improve fuel economy by ≈6.29% while maintaining the terminal state of charge. This study aims to improve the efficiency of action space exploration and optimize the DRL learning strategy, so as to provide a theoretical basis for the design and development of EMS.

A two-level stochastic approach to optimize the energy management strategy for fixed-route hybrid electric vehicles

Technology evolution has brought hybrid electric vehicles to replace fuel-powered vehicles through their emission reduction technology. With the advancement of machine learning technology, an Energy Management Strategy (EMS) is adopted for the real-time optimization of sources like battery, fuel cell, and ultra-capacitor. It also maintains the battery’s state of charge to extend its lifespan while minimizing the fuel consumption of the fuel cell. Furthermore, the stress on the fuel cell and battery is reduced through the optimal utilization of the ultra-capacitor. The ultra-capacitor is implemented for maintaining peak power demands such as during transients, acceleration, or regenerative braking in hybrid electric vehicles. It’s a model-free approach with DC bus voltage. Moreover, deep reinforcement learning is applied so that the Deep Q Network (DQN) agent optimizes the power of the sources. The Centre for Advanced Life Cycle Engineering (CALCE), Proton Exchange Membrane Fuel Cells (PEMFC), and IEEE datasets of battery, fuel cell, and ultra-capacitor respectively, are being used for training and testing. To assess the optimality, a convex optimization technique is employed to verify the results of the energy management strategy. Additionally, the Fuel Cell Hybrid Electric Vehicles (FCHEV) are investigated on public datasets and validated through simulations on different driving cycles like Urban Dynamometer Drive Cycle (UDDS), New European Urban Drive Cycle (NEDC), and Highway Fuel Economy Test Cycle (HWFET). According to the simulations, the data-driven EMS, compared to the traditional model-based EMS, results in lower power fluctuations and optimal efficiency in hybrid electric vehicles.

FlexNet: A warm start method for deep reinforcement learning in hybrid electric vehicle energy management applications

Performance analysis of AI-based energy management in electric vehicles: A case study on classic reinforcement learning

Road Grade Prediction for Predictive Energy Management in Hybrid Electric Vehicles

The optimality-based design of the energy management of a hybrid electric vehicle is a challenging task due to the extensive and complex nonlinear reciprocal effects in the system, as well as the unknown vehicle use in real traffic. The optimization has to consider multiple continuous values of sensor and control variables and has to handle uncertain knowledge. The resulting decision making agent directly influences the objectives like fuel consumption. This contribution presents a concept which solves the energy management using a Deep Reinforcement Learning algorithm which simultaneously permits inadmissible actions during the learning process. Additionally, this approach can include further state variables like the battery temperature, which is not considered in classic energy management approaches. The contribution focuses on the used environment and the interaction with the Deep Reinforcement Learning algorithm.

The increasing adoption of Electric Vehicles (EVs) underscores the growing need for efficient propulsion with Energy Management Systems (EMS). An advanced EMS that integrates Photovoltaic (PV) systems with advanced converters and optimization approaches for a sustainable solution is required for energy management in EVs. Hence, this paper proposes a double cascade boost converter with an ANN-optimized MPPT technique for enhanced efficient energy management in hybrid electric vehicles. The Double Cascade Boost Converter (DCBC) is designed to enhance efficiency by reducing switching losses and providing a high voltage gain. For efficient energy extraction from the PV system under varying solar conditions, an Artificial Neural Network (ANN) optimized Secretary Bird Optimization (SBO) based Maximum Power Point Tracking (MPPT) approach is used. The motor drive uses a 3-phase Voltage Source Inverter (VSI), which converts DC into AC for driving a Brushless Direct Current (BLDC) motor, with a Recurrent Neural Network (RNN) controller employed for precise speed control of the motor. On the Energy Storage System, a Bidirectional DC-DC Converter is integrated for both supercapacitors and storage batteries, providing better predictions of State of Charge (SoC) and battery lifespan of the RNN controller. Simulation outcomes obtained from MATLAB validation illustrate that the proposed work outperforms conventional methodologies in energy conversion efficiency (98.12%) and tracking accuracy (98%).

Recent breakthroughs in artificial intelligence are accelerating the intelligent transformation of vehicles. Vehicle electronic and electrical architectures are converging toward centralized domain controllers. Deep learning, reinforcement learning, and deep reinforcement learning now form the core technologies of domain control. This review surveys advances in deep reinforcement learning in four vehicle domains: intelligent driving, powertrain, chassis, and cockpit. It identifies the main tasks and active research fronts in each domain. In intelligent driving, deep reinforcement learning handles object detection, object tracking, vehicle localization, trajectory prediction, and decision making. In the powertrain domain, it improves power regulation, energy management, and thermal management. In the chassis domain, it enables precise steering, braking, and suspension control. In the cockpit domain, it supports occupant monitoring, comfort regulation, and human–machine interaction. The review then synthesizes research on cross-domain fusion. It identifies transfer learning as a crucial method to address scarce training data and poor generalization. These limits still hinder large-scale deployment of deep reinforcement learning in intelligent electric vehicle domain control. The review closes with future directions: rigorous safety assurance, real-time implementation, and scalable on-board learning. It offers a roadmap for the continued evolution of deep-reinforcement-learning-based vehicle domain control technology.

Adaptive Deep Reinforcement Learning Energy Management for Hybrid Electric Vehicles Considering Driving Condition Recognition