Hierarchical Control Architecture Research Articles

Flexible and Advanced DBS: A distributed control strategy for a DC[sbnd]Nanogrid

Nowadays, it is well known that the energy system is going towards a clear and necessary energy transition. New enabling technologies are needed to accelerate this transition, like the dc nanogrids. The dc nanogrids are designed for a bottom-up approach and play an important role due to their scalability and modularity features. To take advantage of these features, in this paper, the authors propose a flexible and advanced DBS distributed control strategy so to be independent from the dc nanogrid configuration and operating both in stand-alone and in the grid-connected mode. At this scope the dc nanogrid is modelled as the connection of multiple generic nodes and a hierarchical control architecture is applied as a single control structure. It facilitates the seamless transition between the operating modes, independently from the dc nanogrid configuration. The effectiveness of the proposed flexible and advanced DBS is validated by some experiments performed on a testbed integrating a dc nanogrid laboratory prototype.

Coordinating multiple Power-To-Gas plants for optimal management of e-fuel seasonal storage

Seasonal storage is a key feature of future decarbonized energy systems with a high share of renewable energy integration. Power-to-Gas technologies represent a promising solution to enable such storage. They allow the conversion of surplus renewable electricity into e-fuels and their storage in the long-term. Their utilization enables the integration of the electrical, fuel and heating sectors, by converting electricity into fuels and recovering the waste heat from the process. Nevertheless, to design the most profitable management strategy for such systems, advanced control tools are required. This study introduces a novel control architecture for multiple multi-energy systems that share an e-fuel seasonal storage. Each energy system has its own short-term control logic, based on Model-Predictive Control (MPC), which manages day-ahead energy exchanges, while a long-term MPC controller considers yearly dynamics and the system as a whole. This gives additional constraints to the short-term controllers, which ensure the fulfillment of yearly goals. A multi-temporal and multi-spatial hierarchical control architecture is proposed, which enables optimal seasonal storage management, and its operation is verified in a Model-in-the-Loop configuration. The controller efficiently uses seasonal storage to balance seasonal mismatch between production and demand, resulting in higher utilization of renewable energy, lower emissions and costs.

Real-Time Testing Optimal Power Flow in Smart-Transformer-Based Meshed Hybrid Microgrids: Design and Validation

The smart transformer (ST) is a multiport and multi-stage converter that allows for the formation of meshed hybrid microgrids (MHMs) by enabling AC-DC ports in medium and low voltage. This type of microgrid has advantages over the performance of conventional hybrid AC-DC microgrids (HMGs); however, the number of degrees of freedom of the ST increases the complexity of the energy management systems (EMSs), which require adequate and accurate modeling of the power flow of the converters and the MG to find the feasible solution of optimal power flow (OPF) problems in the MHM. An ST’s equivalent power flow model is proposed for formulating the MHM OPF problem and developing low-frequency equivalent models integrated with a decoupled hierarchical control architecture under a real-time simulation approach to the ST-based MHM. A simulation model of the MHM in the Simulink® environment of Matlab® 9.12 is developed and implemented under a digital real-time simulation (DRTS) approach on the OPAL-RT® platform. This model allows for determining the accuracy of the developed equivalent models, both low-frequency and power flow, and determining the MHM performance based on optimal day-ahead scheduling. Simulation test results demonstrated the ST equivalent model’s accuracy and the MHM’s accuracy for OPF problems with an optimal day-ahead scheduling horizon based on the model-in-the-loop (MIL) and DRTS approach.

Deep Reinforcement Learning-Based Hierarchical Energy Control Strategy of a Platoon of Connected Hybrid Electric Vehicles Through Cloud Platform

Due to the features of strong non-linearity and hybrid driving with multi-power sources, a novel deep reinforcement learning(DRL)-based hierarchical energy control architecture is presented for a group of connected hybrid electric vehicles (HEVs) in the platoon through cloud platform. Firstly, a higher-level model predictive control (MPC) law is designed to determine the desired acceleration of connected HEVs in a platoon. Secondly,a reward function with the change of state of charge (SOC) and instantaneous fuel consumption as independent variables is constructed, and the expert knowledge-based deep deterministic policy gradient (DDPG) method with prioritized experience replay (PER) is used for designing the lower-level energy management strategy of connected HEVs platooning. Then, to illustrate the rationality of decision-making by the presented DDPG strategy, the influence of state variables such as vehicle speed, desired acceleration and battery SOC on the agent action values of presented DDPG method is discussed. Finally, the test results show that the proposed control scheme can reasonably allocate the engine and motor powers in real time, and finally achieve the safe and energy-saving driving of connected HEVs in the platoon.

Common direct current (DC) bus integration of DC fast chargers, grid‐scale energy storage, and solar photovoltaic: New York City case study

AbstractThe mass deployment of distributed energy resources (DERs) to achieve clean energy objectives has become a major goal across several states in the U.S. However, the viability and reality of achieving these goals in dense urban areas, such as New York City, are challenged by several ‘Techno‐Economic’ barriers associated with available land space and the number of AC/direct current (DC) conversion stages that requires multiple electrical balance of plant (BOP) equipment for pairing/interconnecting these resources to the grid. The fundamental issue of interconnection is addressed by assessing the use of a common DC bus in a one‐of‐a‐kind configuration (to pair grid‐connected energy storage, photovoltaic, and electric vehicle chargers (EVC) systems) and reduce the number of BOP equipment needed for deployment. Building on similar work that has touched on distribution‐level DC interconnection, this paper will also address the intricacies of interconnecting third‐party and Utility DERs to a DC‐based point of common coupling. It will examine the requisite site controller configuration (control architecture) and requirements to coordinate the energy storage system's use between managing Utility and Third‐Party EVC demand while prioritising dispatch. The result shows that the DC‐coupled system is technologically feasible and hierarchical control architecture is recommended to maintain stability during various use cases proposed. This will inform a lab demonstration of this system that aims to test DC integration of the DERs with recommendations for the microgrid (MG) controllers and reduction in the BOP equipment. These learnings will then be applied to practical grid‐scale deployment of the systems at Con Edison's Cedar Street Substation. This system, if proven successful, has the potential to change the way community distributed generation and MGs are interconnected to the Utility System.

Multi-Scenario Model Predictive Control for Greenhouse Crop Production Considering Market Price Uncertainty

This paper presents a stochastic model predictive control (SMPC) strategy to maximize the economic profit of a greenhouse crop production. The strategy consists of an optimization problem that is solved with a multi-scenario MPC formulation (MS-MPC). It considers the uncertainty of market price by using its historical evolution per year as multiple price scenarios in the cost function. MS-MPC calculates a single set of dates to harvest and sell the crop production that optimizes profits for all the considered scenarios. In addition, MS-MPC determines the optimal temperature references that should be achieved inside the greenhouse for the growth of the crop. A case study for a Mediterranean tomato crop is simulated to analyze the performance of the developed MS-MPC strategy using a hierarchical control architecture with two layers. In the upper layer, MS-MPC calculations are executed following a receding horizon implementation. In the lower layer, regulatory control techniques are applied to reach the optimal temperature references by using natural ventilation and a heating system. Results show that MS-MPC can improve economic profits compared to the use of an average price scenario for the MPC calculations. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This paper was motivated by the need of greenhouse farmers for strategies that maximize profit considering market prices and crop production dynamics. It is not easy for them to make decisions to achieve such long-term objectives while minimizing economic risks, because market prices are very difficult to predict. As a solution, this work presents a novel control strategy to maximize profits by means of automatic selection of the best possible dates to harvest and sell the crop production. This selection is made thanks to considering the uncertainty of market prices in the control strategy by evaluating different scenarios, which are recorded evolutions of the prices from previous years. Although the strategy was tested in simulation, results suggest that using multiple scenarios of historical prices is better for the optimization of profits than just considering an average yearly trend of prices. The more scenarios are considered, the more protection against possible evolution of market prices is obtained. In future research, this control strategy could be extended to include other uncertainties of factors affecting decision making, such as weather forecasts.

Data-Driven-Based Cooperative Resilient Learning Method for Nonlinear MASs Under DoS Attacks.

In this article, we consider the cooperative tracking problem for a class of nonlinear multiagent systems (MASs) with unknown dynamics under denial-of-service (DoS) attacks. To solve such a problem, a hierarchical cooperative resilient learning method, which involves a distributed resilient observer and a decentralized learning controller, is introduced in this article. Due to the existence of communication layers in the hierarchical control architecture, it may lead to communication delays and DoS attacks. Motivated by this consideration, a resilient model-free adaptive control (MFAC) method is developed to withstand the influence of communication delays and DoS attacks. First, a virtual reference signal is designed for each agent to estimate the time-varying reference signal under DoS attacks. To facilitate the tracking of each agent, the virtual reference signal is discretized. Then, a decentralized MFAC algorithm is designed for each agent such that each agent can track the reference signal by only using the obtained local information. Finally, a simulation example is proposed to verify the effectiveness of the developed method.

Drosophila flying in augmented reality reveals the vision-based control autonomy of the optomotor response

For walking, swimming, and flying animals, the optomotor response is essential to stabilize gaze. How flexible is the optomotor response? Classic work in Drosophila has argued that flies adapt flight control under augmented visual feedback conditions during goal-directed bar fixation. However, whether the lower-level, reflexive optomotor response can similarly adapt to augmented visual feedback (partially autonomous) or not (autonomous) over long timescales is poorly understood. To address this question, we developed an augmented reality paradigm to study the vision-based control autonomy of the yaw optomotor response of flying fruit flies (Drosophila). Flies were placed in a flight simulator, which permitted free body rotation about the yaw axis. By feeding back body movements in real time to a visual display, we augmented and inverted visual feedback. Thus, this experimental paradigm caused a constant visual error between expected and actual visual feedback to study potential adaptive visuomotor control. By combining experiments with control theory, we demonstrate that the optomotor response is autonomous during augmented reality flight bouts of up to 30min, which exceeds the reported learning epoch during bar fixation. Agreement between predictions from linear systems theory and experimental data supports the notion that the optomotor response is approximately linear and time invariant within our experimental assay. Even under positive visual feedback, which revealed the stability limit of flies in augmented reality, the optomotor response was autonomous. Our results support a hierarchical motor control architecture in flies with fast and autonomous reflexes at the bottom and more flexible behavior at higher levels.

Predictive cruise control for hybrid electric vehicles based on hierarchical convex optimization

Predictive Cruise Control (PCC) achieves automated driving with improved vehicle economy by incorporating road gradient information from high-precision maps. However, existing research on PCC has primarily centered around conventional gasoline vehicles. Hybrid Electric Vehicle (HEV) offer superior fuel efficiency compared to their traditional counterparts. To further enhance vehicle economy by integrating the PCC system with HEVs, this paper proposes a hierarchical control architecture for HEV-PCC based on convex optimization. Firstly, considering spatial slope information from the road ahead, this paper introduces a Time-Space Integration Model Predictive Control (TSI-MPC) based on the principles of kinetic energy. Secondly, the paper combines a current regression model with the Pontryagin Minimum Principle (PMP) to present a Regression Analysis - Pontryagin Minimum Principle (RA-PMP) approach for the direct analytical solution of the energy management problem. Finally, the strategy is validated through simulation analysis and Hardware-in-Loop (HIL) testing on various real road scenarios. The result indicate that, in comparison to the Equivalent Consumption Minimization Strategy (ECMS) with constant speed cruising (CC), this method achieves an average fuel economy improvement of 9.45%.

Robust Collision-Free Guidance and Control for Underactuated Multirotor Aerial Vehicles

This paper is concerned with the robust collision-free guidance and control of underactuated multirotor aerial vehicles in the presence of moving obstacles capable of accelerating, linear velocity and rotor thrust constraints, and matched model uncertainties and disturbances. We address this problem by using a hierarchical flight control architecture composed of a supervisory outer-loop guidance module and an inner-loop stabilizing control one. The inner loop is designed using a typical hierarchical control scheme that nests the attitude control loop inside the position one. The effectiveness of this scheme relies on proper time-scale separation (TSS) between the closed-loop (faster) rotational and (slower) translational dynamics, which is not straightforward to enforce in practice. However, by combining an integral sliding mode attitude control law, which guarantees instantaneous tracking of the attitude commands, with a smooth and robust position control one, we enforce, by construction, the satisfaction of the TSS, thus avoiding the loss of robustness and use of a dull trial-and-error tweak of gains. On the other hand, the outer-loop guidance is built upon the continuous-control-obstacles method, which is incremented to respect the velocity and actuator constraints and avoid multiple moving obstacles that can accelerate. The overall method is evaluated using a numerical Monte Carlo simulation and is shown to be effective in providing satisfactory tracking performance, collision-free guidance, and the satisfaction of linear velocity and actuator constraints.

Task-oriented human-robot interaction control of a robotic glove utilizing forearm electromyography

Task-oriented myoelectric assistive control of a robotic glove based on forearm electromyography for practical grasping tasks is investigated in this work. Three grasping functions are considered: human intent detection of grasping tasks, grasping mode recognition, and grasping force control. Different combinations of forearm electromyographic signals are adopted for the three functions. Firstly, the overall electromyographic signal is used to trigger the whole grasping task. Secondly, a novel Long Short-Term Memory network is utilized to classify various grasping modes, including pinch grasping and palmar grasping, by analyzing eight-channel electromyographic signals. Thirdly, two-channel proportional control of grasping force using electromyographic flexor/extensor signals is adopted for the robotic glove such that the human hand can relax during the grasping task, while the robotic glove maintains the grasping force. To this end, finite state machine based hierarchical control architecture is proposed for the whole grasping task. Experiments are conducted to validate the proposed task-oriented assistive control method, and the results clearly demonstrates the potential of the proposed method in rehabilitation therapy.

A Study on the Distributed-Control Architecture of a DSP-Based Solid-State Transformer System with Implementation

This article proposes a Distributed-control Architecture (D-CA) and an operation sequence with start-up strategies for a Digital Signal Processor (DSP)-based Solid-State Transformer (SST). Although various control techniques for SSTs have been reported in earlier studies, there is still a lack of research covering comprehensive content, including hierarchical control architectures and operation sequences with start-ups considering the implementation of DSPs. Therefore, this article addresses the following factors of SST. First, the D-CA is described for the design of the hierarchy between control boards. With the D-CA, because sub-boards are in charge of their corresponding DC-link voltage balancing control individually, the computational burden on the master board can be reduced. Second, the operation sequence of the SST system is explained based on the SST with D-CA. The step of DC-link voltage balance is considered throughout the entire operation sequence for safe driving. Furthermore, the PWM start-up strategies for a Cascade H-bridge Multilevel (CHM) converter and Dual Active Bridge (DAB) converter are proposed to prevent switching pulse errors caused by DSP operating characteristics. These start-up strategies reduce the current surges. The validity of the proposed D-CA and operation sequence with start-up strategies are verified by experimental results.

Hierarchical Planning and Control for Box Loco-Manipulation

Humans perform everyday tasks using a combination of locomotion and manipulation skills. Building a system that can handle both skills is essential to creating virtual humans. We present a physically-simulated human capable of solving box rearrangement tasks, which requires a combination of both skills. We propose a hierarchical control architecture, where each level solves the task at a different level of abstraction, and the result is a physics-based simulated virtual human capable of rearranging boxes in a cluttered environment. The control architecture integrates a planner, diffusion models, and physics-based motion imitation of sparse motion clips using deep reinforcement learning. Boxes can vary in size, weight, shape, and placement height. Code and trained control policies are provided.

Multi-Abstractive Neural Controller: An Efficient Hierarchical Control Architecture for Interactive Driving

As learning-based methods make their way from perception systems to planning/control stacks, robot control systems have started to enjoy the benefits that data-driven methods provide. Because control systems directly affect the motion of the robot, data-driven methods, especially black box approaches, need to be used with caution considering aspects such as stability and interpretability. In this paper, we describe a differentiable and hierarchical control architecture. The proposed representation, called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">multi-abstractive neural controller</i> , uses the input image to control the transitions within a novel discrete behavior planner (referred to as the visual automaton generative network, or <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">vAGN</i> ). The output of a vAGN controls the parameters of a set of dynamic movement primitives which provides the system controls. We train this neural controller with real-world driving data via behavior cloning and show improved explainability, sample efficiency, and similarity to human driving.

REVIEW OF CONFIGURATIONS AND STRATEGIES OF CONTROL OF MICRONETWORKS BASED ON POWER ELECTRONICS

The growing implementation of distributed energy resources requires the development of the architecture of traditional electricity systems. With the growing deployment of distributed energy resources, especially for small combined heat and power plants and renewable energy sources based on distributed generation units, the architecture of the future power supply system must be transformed in order to carry out more complex operations. The article provides an overview of micronetwork configurations with greater flexibility and reliability, parameters, structures, and control methods for electronically connected distributed energy resource installations. Of the configurations considered, DC micronetwork and high-frequency AC micronetwork have shown better performance for some special applications, such as low-voltage and geographically small networks. The corresponding parameters in the structures of the blocks of distribution energy resources and the topology of power electronics are considered. With the increase in the nominal capacity of distribution power resource units, multilevel converters provide effective ways to reduce cost and increase efficiency. Methods of controlling electronically connected aggregates of distribution energy resources under different operating modes are described. Under the influence of load and line impedance characteristics, different strategies of load distribution among units of distribution energy resources, control of instantaneous active and reactive power are compared. A multi-level hierarchical control architecture, micronetwork power and energy management is described. A comparison of control architectures based on centralized and distributed technology indicates that the modern micronetwork must become smart and flexible with the help of telecommunication infrastructures and the next generation of information technology.

A Hierarchical Control Scheme for Adaptive Cruise Control System Based on Model Predictive Control

An adaptive cruise control (ACC) system can improve safety and comfort during driving by taking over longitudinal control of the vehicle. It requires the coordination between the upper-layer controller and the lower-layer actuators. In this paper, a hierarchical anti-disturbance cruise control architecture based on electronic stability control (ESC) system is proposed. The upper-layer controller outputs the desired longitudinal acceleration or deceleration to the lower-layer actuators. In order to improve the accuracy of model prediction and achieve the coordinated control of multiple objectives, an upper-layer model prediction cruise controller is established based on feedback control and disturbance compensation. In addition, based on the hydraulic control unit (HCU) model and the vehicle longitudinal dynamics model, a lower-layer nonlinear model predictive deceleration controller is proposed in order to solve the problems of pressure fluctuations and the low accuracy of small decelerations when ESC is used as the actuator for the ACC system. Finally, the simulation and experimental tests were carried out. The results show that the proposed control architecture can improve the stability and comfort of the cruise control process. Moreover, compared with the traditional PID deceleration controller, it effectively improves the deceleration control accuracy.

Assessing the impact of cybersecurity attacks on energy systems

This paper investigates the cyber resiliency of future power systems with high penetration of distributed energy resources using advanced distributed and (or) hierarchical control architectures. Specifically, we simulate cyberattacks on three prototypical use cases, and we identify attack scenarios that are the most damaging to the overall system performance. We show that these attacks can have a significant impact on grid operation. Results provide additional insights into the robustness of the system to the most common cyberattacks.

Status and Development of Research on Orderly Charging and Discharging of Electric Vehicles

As the scale of electric vehicles continues to expand, the charging load of electric vehicles into the network has become an issue that cannot be ignored. This paper introduces the concept and development of ordered charging based on the current background of ordered charging research. The application architecture of ordered charging is summarized, and the advantages and disadvantages of centralized, distributed, and hierarchical control architectures are introduced. The current status of research on orderly charging is analyzed at four levels: steps and methods of load modeling for orderly charging, optimization objectives of orderly charging, optimization methods of orderly charging, and practical projects of orderly charging. The methods of load modeling for orderly charging are summarized, different optimization objectives of grid operation for orderly charging are introduced, and the advantages and disadvantages of different optimization algorithms are compared and analyzed. Practical projects on orderly charging illustrate the great potential of orderly charging. This paper points out four problems of communication, data security, market mechanism, and the number of charging stations that orderly charging is currently facing and proposes feasible solutions. The development prospect of orderly charging being more environmentally friendly, energy-efficient, intelligent, and safe is proposed.

A deep reinforcement learning approach to energy management control with connected information for hybrid electric vehicles

Considering the importance of the energy management strategy for hybrid electric vehicles, this paper is aiming at addressing the energy optimization control issue using reinforcement learning algorithms. Firstly, this paper establishes a hybrid electric vehicle power system model. Secondly, a hierarchical energy optimization control architecture based on networked information is designed, and a traffic signal timing model is used for vehicle target speed range planning in the upper system. More specifically, the optimal vehicle speed is optimized by a model predictive control algorithm. Thirdly, a mathematical model of vehicle speed variation in connected and unconnected states is established to analyze the effect of vehicle speed planning on fuel economy. Finally, three learning-based energy optimization control strategies, namely Q-learning, deep Q network (DQN), and deep deterministic policy gradient (DDPG) algorithms, are designed under the hierarchical energy optimization control architecture. It is shown that the Q-learning algorithm is able to optimize energy control; however, the agent will meet the ”dimension disaster” once it faces a high-dimensional state space issue. Then, a DQN control strategy is introduced to address the problem. Due to the limitation of the discrete output of DQN, the DDPG algorithm is put forward to achieve continuous action control. In the simulation, the superiority of the DDPG algorithm over Q-learning and DQN algorithms in hybrid electric vehicles is illustrated in terms of its robustness and faster convergence for better energy management purposes.

Development of a hierarchical control strategy for a soft knee exoskeleton based on wearable multi-sensor system

Soft exoskeleton is a newly emerging approach for providing assistive forces to the movement of disabled individuals, which has aroused the interest of many researchers due to its promising prospects in rehabilitation training and motion assistance. In this article, a wearable wireless multi-sensor system and a hierarchical control architecture are developed for a soft knee exoskeleton. First, the major components of the soft knee exoskeleton are introduced, including the flexible Bowden-cable actuators, the soft wearable components, and the sensing modules and the control system. The wearable wireless multi-sensor system consists of an inertial measurement unit–based knee flexion–extension angle sensing module, a surface electromyogram–based joint torque estimation module, a miniature pulling force sensing module, and a central communication unit. And then, a three-level hierarchical control strategy—which fuses a high-level motion recognition and torque estimation algorithm, a mid-level knee trajectory generation strategy, and a low-level fuzzy impedance controller—is proposed to realize human motion assistance. Finally, experiments are carried out to verify the effectiveness of the proposed control strategy. The experimental setup is established with the soft knee exoskeleton, a treadmill, a wearable robotic rotary joint with an encoder, and an inelastic rope. The experimental result indicates that the root mean square errors of the joint angle estimation strategy and joint torque estimation strategy are less than 4.3° and 4.8 Nm, while the surface electromyogram–based assistance efficiency and torque-based assistance efficiency under different loading conditions are higher than 10.2% and 18.2%, respectively.