Two-level Hierarchical Control Research Articles

Motion planning in underactuated systems with impulsive phenomenon via dynamic shaping of virtual holonomic constraints

Rhythmic motions are traditionally achieved by developing predetermined paths for the states of the space system to follow. Since these paths are obtained offline, the dynamic behavior fails to adapt to changes in environmental conditions or user command desires. The solution we propose is a new strategy called dynamic shaping, in which the paths are formed online to allow the system to create an orbit with the characteristics we need. Hereupon, this paper focuses on applying this strategy to Dynamics with One Degree of Under-actuation and Impulsive Phenomenon (DSODUIP) to adapt the characteristics of outcomes to be in line with the demands.This research was conducted by leveraging the advantages of virtual holonomic constraints (VHCs) to establish these paths. Therefore, a novel two-level hierarchical control method is designed considering a stability criterion to avoid divergence. At the Low-Level, the controllers stabilize the output of system to follow the VHCs on the system. At the High-Level, the VHCs are modified to shape an orbit with our desired characteristic in the motion. As an illustrative example, the algorithm is implemented to adjust the average angular velocity of a devil stick and the hip velocity of a Three-Link biped robot. Their results vividly demonstrate smooth adjustments and efficient performance in achieving our desired outcomes.

Energy Management Strategy for a Net Zero Emission Islanded Photovoltaic Microgrid-Based Green Hydrogen System

Investing in green hydrogen systems has become a global objective to achieve the net-zero emission goal. Therefore, it is seen as the primary force behind efforts to restructure the world’s energy, lessen our reliance on gas, attain carbon neutrality, and combat climate change. This paper proposes a power management for a net zero emission PV microgrid-based decentralized green hydrogen system. The hybrid microgrid combines a fuel cell, battery, PV, electrolyzer, and compressed hydrogen storage (CHSU) unit aimed at power sharing between the total components of the islanded DC microgrid and minimizing the equivalent hydrogen consumption (EHC) by the fuel cell and the battery. In order to minimize the EHC and maintain the battery SOC, an optimization-based approach known as the Equivalent Consumption Minimization Strategy (ECMS) is used. A rule-based management is used to manage the power consumed by the electrolyzer and the CHSU by the PV system in case of excess power. The battery is controlled by an inverse droop control to regulate the dc bus voltage and the output power of the PV system is maximized by the fuzzy logic controller-based MPPT. As the hybrid microgrid works in the islanded mode, a two-level hierarchical control is applied in order to generate the voltage and the frequency references. The suggested energy management approach establishes the operating point for each system component in order to enhance the system’s efficiency. It allows the hybrid system to use less hydrogen while managing energy more efficiently.

Hierarchical Control of Power Distribution in the Hybrid Energy Storage System of an Ultrafast Charging Station for Electric Vehicles

This paper presents a two-level hierarchical control method for the power distribution between the hybrid energy storage system (HESS) and the main dc bus of a microgrid for ultrafast charging of electric vehicles (EVs). The HESS is composed of a supercapacitor and a battery and is an essential part to fulfill the charging demand of EVs in a microgrid made up of a 220 VRMS ac bus, two dc buses of 600 V and 1500 V, respectively, and four charging points. A state machine defines the four operating modes of the HESS and establishes the conditions for the corresponding transitions among them, namely, charging the battery and the supercapacitor from the bus, injecting the current from the HESS into the 1500 V dc bus to ensure the power balance in the microgrid, regulating the bus voltage, and establishing the disconnection mode. The primary level of the control system regulates the current and voltage of the battery, supercapacitor, and dc bus, while the secondary level establishes the operating mode of the HESS and provides the appropriate references to the primary level. In the primary level, sliding mode control (SMC) is used in both the battery and supercapacitor in the inner loop of a cascade control that implements the standard constant current–constant voltage (CC-CV) charging protocol. In the same level, linear control is applied in the CV phase of the protocol and for bus voltage regulation or the current injection into the bus. PSIM simulations of the operating modes and their corresponding transitions verify the theoretical predictions.

Optimal strategies for large-scale pursuers against one evader: A mean field game-based hierarchical control approach

This paper proposes a two-level hierarchical control approach based on pursuit-evasion game and mean field game for the problem of large-scale pursuers with multi-population against a single evader, which implements that the evader is surrounded by pursuers. At the upper layer, we model the pursuit-evasion game between the centers of the pursuer populations and the single evader, which is formulated as a linear quadratic differential game (LQDG) to obtain the optimal control of each player. Then the optimal trajectories derived from the optimal controls are input to the lower layer as the reference trajectories. At the lower layer, we formulate the tracking of reference trajectories and terminal surrounding to the evader of large-scale pursuers with multi-population as a multi-population mean-field game (MPMFG), which solves the communication and computing difficulties caused by large-scale agents. Then, we derive the variational primal–dual formulation of the proposed MPMFG model and solve it with CA-Net, a coupled alternating neural network approach. Finally, simulation experiments are performed under various pursuit-evasion scenarios, and it is verified that the proposed game-based two-level hierarchical control approach is feasible and effective.

A Distributed Multiparticle Precise Stopping Control Model Based on the Distributed Model Predictive Control Algorithm for High-Speed Trains

Abstract The fixed-point stopping of high-speed trains in stations is generally accomplished through manual operation in China. This situation often leads to a failure to stop at fixed-point signs and causes a fluctuation in the longitudinal acceleration due to the lack of experience of the drivers. To achieve precise, stable, and automatic stopping, a distributed multiparticle precise stopping control model based on the distributed model predictive control (MPC) algorithm is developed in this paper. A two-level hierarchical control structure for the subcontroller of each vehicle is adopted to bring itself to a controlled stop. In the upper control of subcontroller, the MPC algorithm is designed in turn based on the multiparticle mechanism model of the train. In the lower control of subcontroller, the target input from the upper control is converted and distributed. The controlled object, a comprehensive numerical computing model including the spatial dynamic model of the train and its electropneumatic blending braking model, is established and controlled by the corresponding subcontroller and employed to verify the performance of the controller. The influence of the model control parameters on the stopping performance is discussed, and the optimal combination of control parameters is selected. The proposed control model using the optimization parameters is tested and verified through the comprehensive numerical computing model. The results indicate that the stopping error is 0.0075 m, which is much less than the accuracy requirements for fixed-point stopping. The computing time of each subcontroller in real-time is stable at 0.09 s. The coupler impact force between two adjacent vehicles can also be effectively inhibited and eliminated. Its control performance outperforms a proportional–integral–derivative (PID) algorithm. The proposed precise stopping model and comprehensive numerical computing model provide references for the application and algorithm optimization of automatic train operation technology in high-speed trains.

Hierarchical energy management system for multi-microgrid coordination with demand-side management

This paper presents an energy management system (EMS) with demand-side management (DSM) capabilities to optimally coordinate multiple microgrids connected to the same main grid. The proposed EMS comprises a two-level hierarchical control structure, with local controllers at microgrid level, and a supervisory control at the main-grid level. The main contribution of the proposed EMS is that it uses model-based predictive controllers (MPC) and DSM at both levels of the hierarchy. The main-grid level minimizes the imported power from the main grid and the shift from the expected demand for each microgrid. Moreover, it computes optimal aggregated power and demand side management references for each microgrid. The microgrid level manages the energy resources of each microgrid based on the reference signals sent by the main-grid level. Thus, the microgrid level tracks the main-grid level references, while minimizing both the power variation in the battery energy storage system and the variation in shifting factor to manage the demand. A simulation of a multi-microgrid system with three microgrids is implemented to validate the performance of the proposed two-level hierarchical MPC-based EMS with DSM. The proposed control scheme uses fuzzy Takagi–Sugeno models to predict the generation and consumption of the microgrids at both control levels. Furthermore, the proposed configuration is compared with two conventional EMS from the literature. The proposed architecture outperforms previously proposed controllers since, in the considered simulations, it (i) reduces to at least 1/4 the loss-of-power-supply probability for each individual microgrid, (ii) halfs the cycling of the individual storage devices in each microgrid, and (iii) reduces the overall system cost in ∼1.5%.

Safety-Critical and Flexible Cooperative On-Ramp Merging Control of Connected and Automated Vehicles in Mixed Traffic

Cooperative on-ramp merging control for connected and automated vehicles (CAVs) can effectively improve traffic throughput and vehicle fuel efficiency at highway on-ramp merging bottlenecks. However, in the mixed traffic scenario where CAVs and human-driven vehicles (HDVs) coexist, the uncertain maneuvers of human drivers pose a major challenge to merging control in terms of safety and flexibility. To this end, this paper proposes a hierarchical cooperative on-ramp merging control strategy for CAVs to optimize flexible trajectories with safety guarantees in mixed traffic. First, the on-ramp merging control problem for CAVs is considered in the case of a three-vehicle coordination, resulting in an optimal control problem (OCP) coordinating on-ramp and main-lane CAVs for efficient operation while satisfying multiple safety-critical constraints. Second, a two-level hierarchical control architecture is developed to solve the OCP with mixed state-control constraints. The upper-level planner solves an unconstrained OCP with Pontryagin’s Minimum Principle to calculate an expected merging position, which is embedded in the variable time headway of safe merging constraints in the lower-level controller. Then, the controller converts the nonlinear OCP with safety-critical constraints to a quadratic programing (QP) problem by exploiting Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). By solving the QP efficiently, the time and energy efficient trajectory for each CAV is obtained. In addition, a receding horizon control framework is employed, which enables CAVs to determine flexible merging opportunity and tackle the disturbances caused by HDVs. Finally, comprehensive simulation results show that the proposed cooperative on-ramp merging strategy has potential in enabling merging flexibility, improving traffic efficiency and energy economy in real time.

Distributed Energy Resource Control and Utility Interconnection Management: Implementation in a Distributed Microgrid

This article describes the implementation of a brownfield microgrid with the upgrade of existing distributed energy resources (DERs), design of the medium-voltage (MV) interconnection, and implementation of a microgrid control system to meet the U.S. Department of Defense (DOD) needs at Fort Custer, Michigan. Brownfield microgrids involve complexities in design, integration, and commissioning but offer better economic viability due to reduced capital cost upfront. This work tries to identify the challenges associated with brownfield microgrids and provides systematic practical solutions to mitigate them. As part of the upgrades, backup diesel generators were retrofitted to support grid-parallel operation. A battery energy storage system (BESS) was newly installed to manage intermittencies for the existing solar generation and to support the microgrid. The individual DER assets were integrated in a two-level hierarchical microgrid control architecture and controlled in a coordinated manner to achieve seamless islanding and reconnection with the utility grid at the point of common coupling (PCC). The article provides the field data demonstrating the improvement in resiliency through islanded operation as well as maintaining power quality when operating in parallel with the utility. The economic operation of the assets and participation in utility ancillary services were also key goals achieved through the microgrid. This ability allows the military to reduce its total cost of ownership to achieve its mandated energy security needs.

Hierarchical Adaptive Control of the Cyber–Physical Power System With MMC-MTDC for Transient Stability Improvement

Large-scale wind power integration into the power system has promoted the development of a multiterminal DC (MTDC) transmission grid with a modular multilevel converter (MMC). Basically, MTDC with MMC is a typical cyber–physical system with continuous coupling interactions between cyber assets and power systems. However, cyber events may introduce many internet-based vulnerabilities and even result in the loss of transient stability of the power system. Therefore, a voltage compensation-based two-level hierarchical adaptive control strategy is proposed in this article. At the higher level, a modified MMC output current reference calculation method is developed in the αβ framework to guarantee the transient stability of the power system, whereas a feedback adjustment method is proposed in the MMC control framework, at the bottom level, to contain the controller from deviating from its output reference while eliminating the impact of cyber communication delay on transient stability. The article shows that the proposed hierarchical control strategy can improve the transient stability of the power grid under the interference of three-phase ground faults in physical and communication delays in the cyber layer. Finally, the simulation results of the modified IEEE 9-bus test system with MMC–MTDC are used to demonstrate the effectiveness of the proposed scheme.

Fractional order-based hierarchical controller design and evaluation with Bürger-Diehl settler model in a total nitrogen removal wastewater treatment process.

The primary objective of this study was to establish two-level structured control techniques based on the globally known benchmark simulation model no. 1 (BSM1) and the Bürger-Diehl settler model in order to improve effluent quality. The latter was based on the activated sludge model no. 1 (ASM1), while the classic Takacs model was superseded by the more recent Bürger-Diehl settler model with enhanced predictive potential. A two-level hierarchical control structure was considered to maintain the dissolved oxygen concentration basing on the ammonia levels and also a nitrate controller was considered to improve nitrogen removal efficiency. Fractional order PI controllers were considered at the secondary level and advanced control techniques, namely, MPC and fuzzy, were implemented at the primary level. Two advance control schemes, being, FPI-MPC and FPI-fuzzy were designed in the present work. The controllers were designed based on the plant model which was identified using prediction-error minimization method. It was observed that the implemented control strategies in consideration with the plant modifications showed a profound impact in improving the plant performance in terms of the effluent quality. FPI-fuzzy resulted in noticeable 60% and 53% reduction in total nitrogen violations for dry and storm climatic conditions, respectively.

Dynamic modeling and hierarchical control of a concentrated solar power plant with direct molten salt storage

Concentrated solar power (CSP) plant with direct molten salt storage plays an important role in future commercial projects for its high flexibility and reliability. To fully understand its thermodynamic performances and electrical characteristics under various meteorological conditions and load demands, an accurate dynamic model is essential. Although some effort has been devoted to this topic, a complete control system for the plant can hardly be found in the literature. The lack of such information has made the modeling and the analysis of the CSP plant difficult in various applications. This study establishes a detailed CSP plant model and proposes a complete two-level hierarchical control system for a real-life 50 MW linear Fresnel CSP plant located in northwest China. The effectiveness of the developed model has been validated with the real operation data from both static and dynamic perspectives. Then, the plant dynamic behaviors and the performances of the control system under various disturbances are analyzed in detail. The comparison results indicate that the proposed control system shows satisfactory performance (errors within ±1.5%p0 and ±0.5%Pe0) and can effectively inhibit influences of various disturbances. Therefore, the developed model with the proposed control system can serve as a powerful tool for CSP-related research.

Energy Management and Control of a Flywheel Storage System for Peak Shaving Applications

Peak shaving applications provided by energy storage systems enhance the utilization of existing grid infrastructure to accommodate the increased penetration of renewable energy sources. This work investigates the provision of peak shaving services from a flywheel energy storage system installed in a transformer substation. A lexicographic optimization scheme is formulated to define the flywheel power set-points by minimizing the transformer power limit violations and the flywheel energy losses. Convex functions that represent the flywheel power losses and its maximum power are derived and integrated in the proposed scheme. A two-level hierarchical control framework is introduced to operate the transformer-flywheel-system in a way that handles prediction errors and modelling inaccuracies. At the higher level, a model predictive controller is developed that solves the lexicographic optimization scheme using linear programming. At the lower-level, a secondary controller corrects the power set-points of the model predictive controller using real-time measurements. A software platform has been developed for integrating the proposed controllers in an experimental setup to test their effectiveness in a realistic testbed setting, and the flywheel system characteristics are experimentally identified. Simulation and experimental results validate and verify the modeling, identification, control and operation of a real flywheel system for peak shaving services.

TAMPC: A Controller for Escaping Traps in Novel Environments

We propose an approach to online model adaptation and control in the challenging case of hybrid and discontinuous dynamics where actions may lead to difficult-to-escape “trap” states, under a given controller. We first learn dynamics for a system without traps from a randomly collected training set (since we do not know what traps will be encountered online). These “nominal” dynamics allow us to perform tasks in scenarios where the dynamics matches the training data, but when unexpected traps arise in execution, we must find a way to adapt our dynamics and control strategy and continue attempting the task. Our approach, Trap-Aware Model Predictive Control (TAMPC), is a two-level hierarchical control algorithm that reasons about traps and non-nominal dynamics to decide between goal-seeking and recovery policies. An important requirement of our method is the ability to recognize nominal dynamics even when we encounter data that is out-of-distribution w.r.t the training data. We achieve this by learning a representation for dynamics that exploits invariance in the nominal environment, thus allowing better generalization. We evaluate our method on simulated planar pushing and peg-in-hole as well as real robot peg-in-hole problems against adaptive control, reinforcement learning, trap-handling baselines, where traps arise due to unexpected obstacles that we only observe through contact. Our results show that our method outperforms the baselines on difficult tasks, and is comparable to prior trap-handling methods on easier tasks.

Supervisory control configurations design for nitrogen and phosphorus removal in wastewater treatment plants.

Model predictive control (MPC) and Fuzzy controllers are designed in a two-level hierarchical supervisory control framework for control of activated sludge-based wastewater treatment plants (WWTP) in order to efficiently remove nitrogen and phosphorus. Benchmark simulation model No.3 with a bio-phosphorus (ASM3bioP) module is used as a working platform. The hierarchical control framework is used to alter the dissolved oxygen in the seventh reactor (DO7 ) to control ammonia. Lower-level PI, MPC, and Fuzzy are used to control the nitrate levels in the fourth reactor (SNO4 ) by manipulating internal recycle (Qintr ) and DO7 in the seventh tank by manipulating mass transfer coefficient (KL a7 ). MPC and Fuzzy are designed in the supervisory layer to alter the DO7 set-point based on the ammonia composition in the seventh reactor (NH7 ). From the analysis, it is observed that the effluent quality is improved with a decrease in ammonia, TN, and TP. Though a little difference was observed in the cost for all the control strategies, a trade-off is maintained between cost and percentage improvement of effluent quality. MPC-MPC combination showed significant removal in ammonia and better effluent quality when compared to other control strategies. PRACTITIONER POINTS: Developed novel strategies in hierarchical configurations for better nutrient removal with optimal costs in an A2 O process. Lower level control strategies deals with dissolved oxygen in last aeration tank and nitrate in fourth anoxic tank (PI/MPC) Higher level control strategy deals with ammonia in the last aeration tank (MPC/Fuzzy). Average and violations of nutrient removal, economy and overall effluent quality for three weather conditions (Dry, Rain and Strom) are studied. A trade-off is observed between EQI and OCI.

Two-level hierarchical model predictive control with an optimised cost function for energy management in building microgrids

Hybrid energy storage systems have been increasingly envisaged for building microgrids to soften the drawbacks arising from the unpredictability of renewable energy resources and dwelling occupancy. The combination of long- and short-term energy storage systems can enhance the building microgrid capacity of shifting the demand peak toward periods of power generation, increasing the marks of self-consumption rate. However, the design of energy management systems for hybrid energy storage microgrids is more complex than single ones due to a greater number possible solutions. Faced with this issue, this paper proposes a two-level Hierarchical Model Predictive Controller (HMPC) enhanced by two data-driven modules to improve the performance of building microgrids equipped with hybrid energy storage continuously and automatically. With minimum pre-design steps, the two data-driven algorithms improve the accuracy of Li-ion batteries and hydrogen storage models and determine adequate parameters for the HMPC cost function. Relying predominantly on data measurements, the proposed hierarchical controller determines which energy storage device must be run on a daily basis based on the estimation of the annual self-consumption rate and the annual microgrid operation costs. This real-time analysis decreases microgrid expenditure because it avoids grid penalisation regarding the energy autonomy index and reduces the degradation and maintenance of energy storage devices. Compared to a standard rule-based strategy, the proposed controller reduces annual costs up to 5% in residential buildings and 9% in non-residential ones. In contrast, compared to a conventional HMPC the annual expenditure is reduced from 1% to 7% in both types of buildings.

Formalization of the Task of Creating a Mathematical Model of Combined Wastewater Treatment Processes

This paper focuses on the formation approach to formalize the mathematical modeling of wastewater treatment processes for further forming decision support systems for wastewater treatment facilities management on such a theoretical basis. To create an experimental model of formalization of modeling problems, research was conducted on activated sludge from municipal sewage treatment facilities by introducing an oxidant (H2O2) during standard operation of wastewater treatment facilities and introducing a toxicant (sulfur compounds). It was determined that under conditionally standard conditions, the influence of the oxidant is negative: exceeding technological standards of the concentration of dissolved oxygen in water solutions (3.0–13,7 mg/l), low water column transparency (1.4–1.6 cm), higher concentrations of ammonia nitrogen and phosphorus. With the appearance of a toxicant in the form of reduced sulfur compounds (sulfide ions and hydrogen sulfide 1.4–2.8 mg/l), on the contrary, the positive effect of H2O2 on biological water treatment processes was determined: the concentration of dissolved oxygen increases to 3.4 mg/l and the swelling of activated sludge stops. In this case, using a simplified scheme of expert evaluation as a global quality criterion of the biological stage management process of water treatment for rapid assessment of the vitality of activated sludge is justified. As parameters available for direct automatic measurement, it was proposed to use ORP and pH approximated by the regression equation. Also, a conditional scheme of the decision support system for water treatment management was proposed, which will provide two-level hierarchical control: situational and operational in real-time with a preventive response to emergencies; tactical with daily, at least daily, forecasting of the treatment plants.

Two-level hierarchical optimal control for urban traffic networks

A two-level hierarchical optimal control method is proposed in this paper. At the upper level, the reference signals (set-point) are optimized with a data-driven model-free adaptive control (MFAC) method. Traffic signals are regulated with the model predictive control (MPC) with the desired reference signals specified by the upper level. The main contribution is that the set-point optimization is addressed by using the MFAC method which does not need modeling of the traffic system dynamics. Moreover, through this scheme, the planning of vehicle distribution in a road network and the operation of traffic signal control can be executed simultaneously. The effectiveness of the proposed method is testified by an urban traffic network based on MATLAB and VISSIM 9.0 and is also demonstrated by the comparison results with the back-pressure traffic signal control and traditional MPC.

Computational technologies for the synthesis of decentralized control systems for multistage technological processes

This article discusses the development of two-level hierarchical control systems for complex technological processes. This area of research relates to the problems of creating robotic complexes as part of automation systems for managing continuous multi-stage processes according to economic criteria. For the synthesis of decentralized systems of optimal control of processes with a sequential structure, it is recommended to use the dynamic programming method, which allows providing optimal coordinating solutions and justify the setting of local tasks for managing the stages of the technological process, provided that the current states of the input products of these stages are accurately or approximately observed. For an n-stage technological process in conditions of complete information, the problem of a coordinating subsystem is written, the solutions of which do not change for a set of values of input variables of the process and the time period of constancy of external and internal perturbations. It is shown that the implementation of algorithms for controlling individual stages of the process can be performed on the basis of microprocessor technology. Estimates of control error were obtained using a simulation model of a process control system with specified parameters. The article considers the possibility of applying the dynamic programming method in the management of multi-stage technological processes in conditions of incomplete information.

A hierarchical constrained reinforcement learning for optimization of bitumen recovery rate in a primary separation vessel

This work proposes a two-level hierarchical constrained control structure for reinforcement learning (RL) with application in a Primary Separation Vessel (PSV). The lower level is concerned with servo tracking and regulation of the interface level against variances in ore quality by manipulating middlings flow rate. At the higher level, with the objective to optimize bitumen recovery rate, a supervisory interface level setpoint control is implemented. To prevent sanding, tailings density regulation using tailings withdrawal flow rate is proposed. For each case, an asynchronous advantage actor-critic (A3C) based agent is chosen to interact with a high-fidelity PSV model to learn the near optimal control strategy through episodic interactions. Each of the three control loops is sequentially learnt. In the interface level control loop, a behavioral cloning based two-phase learning scheme to promote stable state space exploration is proposed. The proposed hierarchical structure successfully demonstrates improved bitumen recovery rate by manipulating the interface level while preventing sanding.

Geometric Region-Based Swarm Robotics Path Planning in an Unknown Occluded Environment

This article presents a geometrical region-based shape control methodology for navigating a cohesive swarm-robotic structure toward the goal even in a field occluded by unknown obstacles. In this control approach, initially, the robotic swarm is conceived to lie within a well-defined virtual circular region thus preserving a strict interagent cohesiveness among them. However, during the progression, for evading severely constricted obstacles, the virtual circle has been allowed to change its shape and in the process, varied elliptical shapes are made to evolve. In essence, for a collision-free solution, this shrinking aspect (from circle to ellipse) depends entirely on the number of agents in the swarm and at the same time also reliance on the sensed distance between two nearest obstacles through which the shrunken circle or the virtual ellipse will be able to pass. Consequently, shape switching is a dynamic as well as a stochastic process throughout the journey of the swarm. For achieving these objectives, a two-level hierarchical control strategy has been employed. Moreover, during aggregating toward the target, the actuation failure of any agent or agents may occur. In this perspective, the proposed control law has been updated adaptively throughout the route such that agent failure does not encumber the mission. Finally, the extensive simulation results along with the hardware experimentation are provided to demonstrate the efficacy of the proposed scheme.