Energy Management Task Research Articles

Optimal Rule-Interposing Reinforcement Learning-Based Energy Management of Series—Parallel-Connected Hybrid Electric Vehicles

P2–P3 series–parallel hybrid electric vehicles exhibit complex configurations with multiple power sources and operational modes, presenting a difficulty in developing efficient energy management strategies. This paper takes a P2–P3 series–parallel hybrid power system-KunTye 2DHT system as the research object and proposes a deep reinforcement learning framework based on pre-optimized energy management to improve the energy consumption performance of the hybrid electric vehicles. Firstly, a control-oriented model is established based on its system configuration and characteristics. Then, the optimal distribution of the motor energy under different operating modes is pre-optimized, which aims to reduce the energy management task’s dimensionality by equating two motors as an equivalent motor. Subsequently, based on real-time traffic information under connected conditions, deep reinforcement learning is utilized to optimize the optimal operating modes of the hybrid system and the optimal distribution between the engine and equivalent motors. Combining the pre-optimized results, the optimal energy distribution between the engine and the two motors in the system is achieved. Finally, performance comparisons are made between the predictive control and the traditional Dynamic Programming and Adaptive Equivalent Consumption Minimization Strategy, revealing the proposed optimization algorithm’s promising potential in reducing fuel consumption.

Pre-optimization-assisted deep reinforcement learning-based energy management strategy for a series–parallel hybrid electric truck

P2-P3 series-parallel hybrid electric vehicles (HEVs) feature intricate topologies with multiple power sources and multiple working modes, posing a challenge for developing effective energy management strategies (EMSs). This paper introduces a framework that combines deep reinforcement learning (DRL) with pre-optimized energy management to address this challenge. Considering the characteristics of HEVs, the framework incorporates the pre-optimized equivalent motor and pre-optimized mode selection rule into a deep deterministic policy gradient (DDPG) loop. The pre-optimization of the equivalent motor aims to reduce the energy management task’s dimensionality by equating two motors as the equivalent motor, while the pre-optimized mode selection rule systematically integrates mode selection and energy allocation into the EMS. The study assumes HEVs operate in connected urban environments and incorporates middle-horizon traffic information to enhance EMS performance. Additionally, the study integrates the frequent engine start-stop characteristics of HEVs into EMS design. Simulation results demonstrate that the fuel economy of the proposed EMSs in this study ranges from 87.2% to 90.7% of the DP-based EMS. The inclusion of an engine start-stop penalty significantly reduces the number of engine start-stops without compromising fuel economy.

Enabling intelligent transferable energy management of series hybrid electric tracked vehicle across motion dimensions via soft actor-critic algorithm

Due to the complex driving conditions faced by hybrid electric tracked vehicles, energy management is crucial for improving fuel economy. However, developing an energy management strategy (EMS) is a time-consuming and labor-intensive task, which is challenging to generalize across different driving tasks. To solve this problem and shorten the development cycle of EMSs, this article proposes a novel transferable energy management framework for a series hybrid electric tracked vehicle (SHETV) across motion dimensions. To fully reuse the learned knowledge from longitudinal motion into both longitudinal and lateral motion, this framework merges transfer learning (TL) into the state-of-the-art deep reinforcement learning (DRL) algorithm, soft actor-critic (SAC), to formulate a novel deep transfer reinforcement learning (DTRL) method, with the transfer of both the neural networks and the pre-trained experience replay buffer. Simulation results indicate that the proposed EMS accelerates the convergence speed by 75.38%, enhances the learning ability by 19.05%, and improves the fuel economy by 5.08% compared to the baseline EMS. This article contributes to correlating different energy management tasks and reusing the existing EMS for the rapid development of a new EMS of the hybrid electric tracked vehicle.

A Three-Transistor Energy Management Circuit for Energy-Harvesting-Powered IoT Devices

Energy harvesting (EH) provides a promising solution for powering distributed Internet of Things (IoT) devices. Due to the low-level and sporadic ambient energy supply, an EH-powered device should operate in an intermittent and energy-driven mode. Commercial voltage supervisors were not optimized for the EH scenario, making it difficult to satisfy all new demands. The conventional energy management (EM) circuit has a risk of locking up during the turn-on transient; therefore, it might fail to power the IoT load device. Previous technologies have used a relatively large circuit to solve this problem. In this paper, a concise discrete three-transistor energy management circuit (3T-EM) is proposed. It can track stored energy, switch on/off to the load device, and provide a regulated voltage output. These key functions are realized by utilizing a minimum number of components; therefore, power consumption and manufacturing cost are largely cut. The voltage thresholds and minimum input current are theoretically derived. In experiments, the on/off thresholds can be adjusted accurately, as predicted by the theory. The 3T-EM circuit can ensure the correct operation when the input current is as low as 0.4 μA. Control experiments also prove the effectiveness and performance of the 3T-EM circuit. The proposed 3T-EM circuit shows the characteristics of low cost, low power, inherent regulation, high voltage rating, and good predictability. It is a good candidate to perform the EM task in widely distributed EH-powered IoT devices.

Model-free data-driven approach assisted Deep Reinforcement Learning for Optimal Energy Management in MicroGrid

Uncertainties of solar PhotoVoltaic (solar PV) generation and Electric Vehicle (EV) demand are major issues for Optimal Energy Management (OEM) tasks in MicroGrid (MG), especially regarding power system stability and increased overall demand. A Probabilistic Load Flow (PLF) with a Battery Energy Storage System (BESS) controlled by an Energy Management System (EMS) can deal with these issues. However, the PLF and optimization algorithm based on the iterative method leads to a high computation burden. Therefore, this paper proposes a model-free data-driven approach assisted Deep Reinforcement Learning (DRL) to decrease the computation burden of PLF and problem-solving. Deep Neural Networks (DNNs) are developed as a model-free data-driven to estimate the power flow parameters of MG instead of PLF. Moreover, the DRL named a Deep Deterministic Policy Gradient (DDPG) is deployed as the optimization algorithm to find the optimal solution in the OEM task. In addition, finding appropriate parameters of the DDPG is proposed in this paper. To showcase the efficacy of the proposed method, a low-voltage distribution network is developed as the MG. Objective functions including exchanged energy, carbon emission, and BESS degradation costs are considered in this work. Simulation results indicate that the proposed method is capable of reducing the computation burden by 88.47% in comparison to the Differential Evolution (DE) algorithm that uses 10 populations. Moreover, the best agent model obtained from finding suitable parameters of DDPG can provide a total cost of 115.63 USD/day, which is the lowest cost compared with the cost obtained by the DE.

Spacecraft energy management using convex optimisation

This paper considers a spacecraft energy management strategy for optimally scheduling the power available for secondary tasks, such as communications and scientific payloads, while meeting the power demand for primary tasks, such as thermal management and propulsion, and adhering to operating constraints. A solution based on convex optimisation and model predictive control is proposed. Mathematical models of the various energy subsystems are used to formulate the energy management task over a future horizon as a constrained optimisation problem. We show that this problem can be formulated as a convex program and we derive an energy management strategy in the form of a model predictive control law. The convex problem formulation guarantees global optimality of the presented solution and opens the way for future onboard real-time implementations.

Boosting or nudging energy consumption? The importance of cognitive aspects when adopting non-monetary interventions

Identifying effective behaviour-change interventions to promote energy conservation in the residential sphere has been the topic of extensive empirical research. While existing literature has advised several successful interventions, their context-dependency is still an open question. Furthermore, existing evidence has primarily focused on trialling nudges, that is, interventions that influence behaviour directly by changing aspects of the decision environment and circumventing cognitive bias. Boosts, which instead aim to influence behaviour by fostering the competences of decision-makers and correcting bias, are still under-researched in this domain.We present the results of an online experiment where we compare the effects of a nudge-like, and a boost-like intervention on decisions in an incentive-compatible energy management task. These interventions are trialled in relatively high income and low income populations. Finally, we repeat the experiment with the same participants after removing the interventions.Our results show that income is a significant determinant of performance in the task, with the higher income cohort performing better than the lower income counterpart. However, this difference is largely explained by underlying idiosyncratic factors, namely the level of cognitive competences of participants. Furthermore, both boosting and nudging approaches brought energy savings close to the ceiling of achievable goals, but the boosting approach proved more challenging for participants with lower cognitive competences. Finally, we report evidence of intertemporal spillovers.We conclude by highlighting directions of future research to further assess the interplay between intervention choice and cognitive aspects in the field, to design effective behaviour-change policies in an ethical and targeted manner.

A Stochastic-IGDT model for energy management in isolated microgrids considering failures and demand response

In power systems, contingencies and outages are more frequent nowadays due to climate changing effects. This circumstance along equipment aging may lead to unexpected failures and outages in power and energy systems. This issue is especially critical in isolated microgrids, which must be supplied by means of own resources and onsite assets. In these systems, unexpected failures may notably provoke a detriment of the economy and users’ satisfaction. In order to minimize the impact of these incidents, this paper proposes a novel energy management tool for isolated microgrids that are robust against failures. To this end, a novel stochastic-IGDT formulation is developed, by which typical uncertainties are comfortably modelled via scenarios while components’ failures are treated in a robust fashion using IGDT. A solution procedure is proposed in which the operation cost is also considered in order to reproduce useful results and limit the cost of reliability. A variety of simulations are conducted in order to validate the developed model and discuss the particularities derived from considering failures in the energy management task. Moreover, the role of demand response programs is also foregrounded. In particular, the demand response programs allow reducing the operation costs by 3 % while the scheduling result admits up to 13 h more of accumulated failures, thus confirming a positive effect of such initiatives in both economy and robustness against failures.

Assessing demand compliance and reliability in the Philippine off-grid islands with Model Predictive Control microgrid coordination

This paper considers off-grid microgrids (MGs) from the Philippine archipelago and analyses their energy generation in differents aspects. Seven different energy clusters are used, representing realistic configurations and renewable energy shares. A Robust Model Predictive Control (MPC) framework is used for the energy management and coordination task of these island MGs. The MPC is based on a min./max. optimization procedure, which takes into account the whole uncertainty set. The reliability of the MG operations are analysed with respect to the different clusters; this evaluation is conducted using μ-analysis, performed with respect to the baseline model and the uncertainty set. The demand-side compliance of the MG is also investigated, with respect to stochastic behaviours of the demands and of the renewable sources (wind and solar). Numerical simulation results are presented in order to demonstrate that reliable power outlets are produced despite variation in renewables and of the demands. This paper offers a thorough analysis of simple energy system coordinated via MPC, showing how this method can indeed be used for renewable MG management, offering robustness and ensuring reliability.

HIL Evaluation of a Novel Real-time Energy Management System for an HEV with a Continuously Variable Transmission

One of the most important challenges facing automotive engineers is reducing vehicle fuel consumption and improving the drivability index. Modern hybrid electric powertrains play an important role in reducing fuel consumption. Continuously variable transmission (CVT) is an automatic transmission that can change the gear ratio seamlessly using a belt and pulleys. CVT performs with infinite gear ratios. Controlling and determining the optimal gear ratio, especially in a complex hybrid powertrain, is a major challenge. Therefore, a multi-parametric model predictive controller with real-time implementation capability is proposed that can handle the energy management task concurrently with gear shifting strategy in a parallel pre-transmission hybrid electric vehicle. The proposed controller hardware-in-the-loop (HIL) validation procedure on the high-fidelity Autonomie model shows significant improvement in fuel economy while maintaining drivability in three different driving schedules. HIL evaluation guarantees the real-time capability and the proposed controller readiness to be implemented to real-world control hardware.

Energy-Net: A Deep Learning Approach for Smart Energy Management in IoT-Based Smart Cities

Although intelligent load forecasting is essential for optimal energy management (EM) in smart cities, there is a lack of current research exploring EM in well-regulated Internet-of-Things (IoT) networks. This article develops a new deep learning (DL) model for efficient forecasting of short-term energy consumption while maintaining effective communication between energy providers and users. The proposed Energy-Net stack comprises multiple stacked spatiotemporal modules, where each module consists of a temporal transformer (TT) submodule and a spatial transformer (ST) submodule. The TT models the temporal relationships in load data; and the ST submodule extracts hidden spatial information by integrating convolutional layers and includes an improved self-attention mechanism. The experimental evaluation on IHPEC and independent system operator New England (ISO-NE) data set demonstrates the superiority of Energy-Net over recent cutting-edge DL models with root mean-square error (RMSE) of 0.354 and 0.535, respectively. The computational complexity of Energy-Net is appropriate for dependable resource-constrained IoT devices (i.e., fog nodes or edge nodes) linked to a joint IoT-cloud server that interacts with connected smart grids to handle EM tasks.

Energy-aware task allocation strategy for multi robot system

ABSTRACT This paper proposes a distributed energy management task allocation strategy for a multi-robot system (MRS) involved in goods transportation. The main goal in such type of application is maximizing the productivity by transporting the maximum of goods. However, an important aspect of the optimal use of an MRS is an efficient energy management. The aim of this work is to develop a distributed cooperative strategy that considers energy management in task allocation. For this purpose, an Iterative Local Search-based (ILS) metaheuristic and a novel heuristic-based utility function are developed to solve the multi-robot task allocation problem. The proposed strategy uses the MRS as a population of individuals to solve the task allocation in a parallel distributed manner. Each robot runs the ILS-based algorithm and provides its solution for the task allocation, due to the fact that some of ILS parameters are randomly chosen. Thus, this strategy combines the exploration advantage of the population-based metaheuristics and the exploitation advantage and easy implementation on embedded systems (e.g., MRS) of the single-point-search metaheuristics. The proposed strategy is evaluated in a simulated environment and compared to three heuristics, in terms of the average consumed energy, the productivity of the system, and productivity per energy unit consumed.

Fully-Convolutional Denoising Auto-Encoders for NILM in Large Non-Residential Buildings

Great concern regarding energy efficiency has led the research community to develop approaches which enhance the energy awareness by means of insightful representations. An example of intuitive energy representation is the parts-based representation provided by Non-Intrusive Load Monitoring (NILM) techniques which decompose non-measured individual loads from a single total measurement of the installation, resulting in more detailed information about how the energy is spent along the electrical system. Although there are previous works that have achieved important results on NILM, the majority of the NILM systems were only validated in residential buildings, leaving a niche for the study of energy disaggregation in non-residential buildings, which present a specific behavior. In this article, we suggest a novel fully-convolutional denoising auto-encoder architecture (FCN-dAE) as a convenient NILM system for large non-residential buildings, and it is compared, in terms of particular aspects of large buildings, to previous denoising auto-encoder approaches (dAE) using real electrical consumption from a hospital facility. Furthermore, by means of three use cases, we show that our approach provides extra helpful funcionalities for energy management tasks in large buildings, such as meter replacement, gap filling or novelty detection.

Microgrid Energy Management With Asynchronous Decentralized Particle Swarm Optimization

Controlling distributed energy resources (DERs) in low voltage microgrids is a challenging task for operators. The simultaneous operation of independent small-scale DER owners could compromise the operator’s hierarchical and centralized control to reach system stability and cost optimization. Recent Decentralized Energy Management (DEM) approaches provide flexibility for DERs control, but several existing solutions depend on powerful and expensive computer clusters and their ability to deal with a high burden of data in the communication channel. This work is motivated towards a DEM framework that involves independent DER owners while microgrid operator still maintains a hierarchical control philosophy. The framework must include a method to reduce the need of powerful computer clusters and depend on low bandwidth communications channel. Here, a multi-layered framework for every DER, consisting of physical, control, and agent layers for DEM is approached, where the agent layer participates in the energy management task. An Asynchronous Decentralized PSO (ADPSO) algorithm is proposed for the agent layer based on its primal characteristic: it can reach a consensus state between networked computing units by exchanging asynchronously only the state variable through the communications channel. The proposed solution allows the integration of DEM capabilities within the physical controller of the DERs, distinguishing it from other decentralized solutions. Easiness of implementation and low computational requirements are shown by performing DEM tests on single board computers. The tests show improved convergence rate, improved swarm diversity behavior and fast consensus reaching of DEM optimization.

Probabilistic Load Forecasting Based on Adaptive Online Learning

Load forecasting is crucial for multiple energy management tasks such as scheduling generation capacity, planning supply and demand, and minimizing energy trade costs. Such relevance has increased even more in recent years due to the integration of renewable energies, electric cars, and microgrids. Conventional load forecasting techniques obtain single-value load forecasts by exploiting consumption patterns of past load demand. However, such techniques cannot assess intrinsic uncertainties in load demand, and cannot capture dynamic changes in consumption patterns. To address these problems, this paper presents a method for probabilistic load forecasting based on the adaptive online learning of hidden Markov models. We propose learning and forecasting techniques with theoretical guarantees, and experimentally assess their performance in multiple scenarios. In particular, we develop adaptive online learning techniques that update model parameters recursively, and sequential prediction techniques that obtain probabilistic forecasts using the most recent parameters. The performance of the method is evaluated using multiple datasets corresponding with regions that have different sizes and display assorted time-varying consumption patterns. The results show that the proposed method can significantly improve the performance of existing techniques for a wide range of scenarios.

Fault-tolerant energy management for an industrial microgrid: A compact optimization method

This work presents an optimization-based control method for the fault-tolerant energy management task of an industrial energy microgrid, based on a sugarcane power plant. The studied microgrid has several renewable energy sources, such as photovoltaic panels, wind turbines and biomass power generation, being subject to different operational constraints and load demands. The proposed management policy guarantees that these demands are met at every sampling instant, despite eventual faults. This law is derived from the solution of an optimization problem that combines the formalism of a Moving Horizon Estimation (MHE) scheme (to estimate faults) and a Model Predictive Control (MPC) loop (for fault-tolerant control goals); it chooses which energy source to use, seeking maximal profit and increased sustainability. The predictive controller part of the scheme is based on a linear time-varying model of the process, which is scheduled with respect to the fault estimation brought up by the MHE. Via numerical simulations, it is demonstrated that the proposed method, when compared to other MPC strategies, exhibits enhanced performances.

A hybrid deep learning-based method for short-term building energy load prediction combined with an interpretation process

Data driven-based building energy load prediction is of great value for building energy management tasks such as fault diagnosis and optimal control. However, there are two challenges for conventional data driven-based prediction methods. The first challenge is that time-lag measurements such as historical cooling loads still cannot be taken full advantage of. To deal with this challenge, a hybrid prediction method is proposed based on long short-term memory networks and artificial neural networks. The second challenge is that data driven-based models are hard to explain by domain knowledge. To deal with this challenge, an interpretation method is proposed based on a dimensionless sensitivity index and a weighted Manhattan distance. Operation data of a public building are utilized to evaluate the proposed methods. Results show that the proposed hybrid prediction method has higher prediction accuracy than conventional prediction methods in one-hour-ahead cooling load prediction. Crucial factors affecting building cooling loads are revealed successfully based on the proposed sensitivity index. Moreover, the weighted Manhattan distance is utilized to quantify the difference between predicted conditions and known conditions of training data. Results show that the prediction accuracy of data driven-based methods is reduced with the increase of the weighted Manhattan distance. It is further discovered that relationships between logarithmic prediction residuals and corresponding logarithmic weighted Manhattan distances are approximatively linear.

Persistent Crowd Tracking Using Unmanned AerIal Vehicle Swarms: A Novel Framework for Energy and Mobility Management

Aerial surveillance systems are a cost-effective and on-demand solution for smart city monitoring. However, the deployment of a swarm of unmanned aerial vehicles (UAVs) for continuous video capture of mobile ground target (MGT) activities poses new research issues in terms of energy management, scenario coverage, and multidevice task coordination. In this article, we address these challenges by proposing Persistent Crowd Tracking Using UAVs (PERCEIVE), a novel framework for continuous video surveillance via UAVs that are periodically replenished through mobile charging stations (MCSs).

Models for predicting damage due to accidents at energy objects and in energy systems of enterprises

Achieving energy security by preventing and timely eliminating the consequences of accidents at energy facilities and in energy supply systems of enterprises is one of the important tasks of energy management. The basis for planning appropriate energy security measures is the prediction of damage from these accidents. The purpose of forecasting is to assess the possibility of an accident occurring at some point in time and leading to a particular damage, and to assess the magnitude of this damage. The article proposed methodological approaches to the construction of mathematical models of such prediction. In this case, as an indicator of damage, the economic losses caused by these accidents are taken. The simulation is based on the representation of this indicator in the form of a step change function of the magnitude of losses in the event of an accident. Depending on the amount of information available in the period prior to forecasting, the mathematical representation of the forecasting problem is reduced to the construction of conditionally determined or stochastic models. Conditionally determined models allow obtaining acceptable damage estimates with a short period of retrospection and small amounts of information, and stochastic models with significantly large amounts. At the same time, the principle of “maximum uncertainty” formalized in the form of maximum entropy is the basis for removing uncertainty in the construction of both conditionally determined and stochastic models. Its use has allowed increasing the objectivity of forecasts by minimizing the subjective information used in modeling. The proposed approaches to the construction of mathematical models for predicting accidents at energy facilities and power supply systems of enterprises are the basis for creating specific techniques for solving relevant energy management tasks both at the micro level at the scale of individual enterprises and at the macro level at the scale of industries, regions and the state as a whole.

Online Energy Management and Heterogeneous Task Scheduling for Smart Communities with Residential Cogeneration and Renewable Energy

With the development of renewable energy technology and communication technology in recent years, many residents now utilize renewable energy devices in their residences with energy storage systems. We have full confidence in the promising prospects of sharing idle energy with others in a community. However, it is a great challenge to share residents’ energy with others in a community to minimize the total cost of all residents. In this paper, we study the problem of energy management and task scheduling for a community with renewable energy and residential cogeneration, such as residential combined heat and power system (resCHP) to pay the least electricity bill. We take elastic and inelastic load demands into account which are delay intolerant and delay tolerant tasks in the community. The minimum cost problem of a non-cooperative community is extracted into a random non-convex optimization problem with some physical constraints. Our objective is to minimize the time-average cost for each resident in the community, including the cost of the external grid and natural gas. The Lyapunov optimization theory and a primal-dual gradient method are adopted to tackle this problem, which needs no future data and has low computational complexity. Furthermore, we design a cooperative renewable energy sharing algorithm based on State-action-reward-state-action (Sarsa) Algorithm, in the condition that each residence in the community is able to communicate with its neighbors by a central controller. Finally, extensive simulations are presented to validate the proposed algorithms by using practical data.