Hierarchical Control Approach Research Articles

Interactive learning for multi-finger dexterous hand: A model-free hierarchical deep reinforcement learning approach

When a multi-fingered dexterous hand interacts with the external environment, it encounters various challenges, including the utilization of complex control techniques and the intricate coordination of finger motion sequences. Previous studies have primarily concentrated on investigating the interaction between multi-fingered dexterous hands and external objects, usually using model-based control or model-free reinforcement learning techniques. However, during practical implementation, various constraining factors are encountered, such as intricate modeling and limited interaction capabilities. In practical scenarios, the utilization of multi-fingered dexterous hands is imperative for the swift and efficient execution of a wide range of interactive tasks, including but not limited to throwing a ball and playing rock-paper-scissors. These tasks require skilled manual dexterity to demonstrate both precise control and quick responsiveness. To tackle this issue, we propose a hierarchical control approach for multi-fingered dexterous hands with interactive functionalities, utilizing model-free deep reinforcement learning. The complex interaction task is decomposed into simple sub-tasks using hierarchical strategy and action primitive decomposition, which effectively reduces the complexity of the action space, and achieves the motion planning and end finger trajectory control of dexterous hand. In a simulated environment, the aforementioned method has successfully executed interactive tasks, including ball throwing and playing rock-paper-scissors. It achieved a maximum normalized reward of 0.83 and an 84 % success rate. These results are noteworthy in terms of both control accuracy and response speed. This study offers novel insights into the effective resolution of the intricate challenges associated with interactions involving multi-fingered dexterous hands and human-computer interaction.

Predefined-time distributed optimization and anti-disturbance control for nonlinear multi-agent system with neural network estimator: A hierarchical framework

This paper addresses the predefined-time distributed optimization of nonlinear multi-agent system using a hierarchical control approach. Considering unknown nonlinear functions and external disturbances, we propose a two-layer hierarchical control framework. At the first layer, a predefined-time distributed estimator is employed to produce optimal consensus trajectories. At the second layer, a neural-network-based predefined-time disturbance observer is introduced to estimate the disturbance, with neural networks used to approximate the unknown nonlinear functions. A neural-network-based anti-disturbance sliding mode control mechanism is presented to ensure that the system trajectories can track the optimal trajectories within a predefined time. The feasibility of this hierarchical control framework is verified by utilizing the Lyapunov method. Numerical simulations are conducted separately using models of robotic arms and mobile robots to validate the effectiveness of the proposed method.

Wheel Torque Distribution Control Strategy for Electric Vehicles Dynamic Performance With an Electric Torque Vectoring Drive Axle

The ability of active chassis control for dynamic performance improvement of electric vehicles with centralized drive system is limited duo to the driving torque between two-side wheels always equally distributed. To solve this problem, this paper proposes a novel electric torque vectoring (TV) drive-axle and its torque distribution control strategy, which can realize the function of arbitrary distribution of the left/right wheel driving torque in the form of centralized drive, thereby improving the dynamic performance of the vehicle. First, the structure and torque distribution principle of TV drive-axle are analyzed theoretically. Subsequently, an accurate mathematical model of TV drive-axle is established based on bond graph theory. Next, a wheel torque distribution control strategy of TV drive-axle is proposed based on hierarchical control approach to improve the vehicle’s handling stability. Wherein, the target yaw rate motion tracking controller as the upper-level controller is designed based on the second-order super-twist sliding mode control (SS-SMC) algorithm, and the wheel drive anti-slip controller in lower-level is used to restrict possible wheels slipping. Finally, the off-line simulation and hardware-in-the-loop (HIL) experiment results suggest that the proposed novel TV drive-axle and control strategy can significantly improve the handling stability and steering flexibility of the vehicle.

Predefined-Time Bipartite Time-Varying Formation Tracking Control of Networked Autonomous Surface Vehicles Via Hierarchical Control Approach

Predefined-Time Bipartite Time-Varying Formation Tracking Control of Networked Autonomous Surface Vehicles Via Hierarchical Control Approach

A hierarchical control approach for virtual coupling in metro trains

AbstractAs an emerging technology, virtual coupling improves the efficiency and flexibility of metro services by forming multiple trains (units) as a virtually coupled train set (VCTS) without mechanical couplers. However, to realize the desired VCTS operation in practical metro services, a significant gap to be filled is that the implicit and nonlinear safety constraints are hard to be addressed in real‐time control. Thus, this paper proposes a hierarchical control approach with a two‐layer framework. In the upper layer, a trajectory planning method is designed, which addresses the safety constraints and prescribes a reference trajectory for each unit. In the lower layer, a model predictive control approach is constructed, by which each unit accurately tracks its reference trajectory in real time. Afterward, the proposed hierarchical control approach is employed to realize VCTS inter‐station operation in field tests. The average tracking error of speed is around 8 cm/s and the stopping error is less than 12 cm. The following distance between two units in VCTS is reduced to less than 105 m at the speed of 80 km/h, which is a breakthrough in the development of VCTS.

Optimal strategies for large-scale pursuers against one evader: A mean field game-based hierarchical control approach

This paper proposes a two-level hierarchical control approach based on pursuit-evasion game and mean field game for the problem of large-scale pursuers with multi-population against a single evader, which implements that the evader is surrounded by pursuers. At the upper layer, we model the pursuit-evasion game between the centers of the pursuer populations and the single evader, which is formulated as a linear quadratic differential game (LQDG) to obtain the optimal control of each player. Then the optimal trajectories derived from the optimal controls are input to the lower layer as the reference trajectories. At the lower layer, we formulate the tracking of reference trajectories and terminal surrounding to the evader of large-scale pursuers with multi-population as a multi-population mean-field game (MPMFG), which solves the communication and computing difficulties caused by large-scale agents. Then, we derive the variational primal–dual formulation of the proposed MPMFG model and solve it with CA-Net, a coupled alternating neural network approach. Finally, simulation experiments are performed under various pursuit-evasion scenarios, and it is verified that the proposed game-based two-level hierarchical control approach is feasible and effective.

Hierarchical false discovery rate control for high-dimensional survival analysis with interactions

With the development of data collection techniques, analysis with a survival response and high-dimensional covariates has become routine. Here we consider an interaction model, which includes a set of low-dimensional covariates, a set of high-dimensional covariates, and their interactions. This model has been motivated by gene-environment (G-E) interaction analysis, where the E variables have a low dimension, and the G variables have a high dimension. For such a model, there has been extensive research on estimation and variable selection. Comparatively, inference studies with a valid false discovery rate (FDR) control have been very limited. The existing high-dimensional inference tools cannot be directly applied to interaction models, as interactions and main effects are not “equal”. In this article, for high-dimensional survival analysis with interactions, we model survival using the Accelerated Failure Time (AFT) model and adopt a “weighted least squares + debiased Lasso” approach for estimation and selection. A hierarchical FDR control approach is developed for inference and respect of the “main effects, interactions” hierarchy. The asymptotic distribution properties of the debiased Lasso estimators are rigorously established. Simulation demonstrates the satisfactory performance of the proposed approach, and the analysis of a breast cancer dataset further establishes its practical utility.

A three-layer hierarchical model-based approach for network-wide traffic signal control

This paper introduces a three-layer hierarchical model-based approach for network-wide traffic signal control, which consists of a region coordination layer, an intersection coordination layer and a local signal control layer. The hierarchical control problem is formulated by two model predictive control (MPC) schemes. The upper-level control deals with the region coordination problem, which considers the cycle length optimization for different regions based on the macroscopic fundamental diagram. Under the guidance of the upper level MPC, a distributed MPC is developed to address both the green split optimization at each local intersection and the intersection coordination. For computational efficiency, the network is decomposed into separated intersections while the Lagrangian algorithm is developed for intersection coordination. Experimental results on the Nguyen-Dupuis network prove that our proposed three-layer hierarchical control approach has great potential in reducing the computational complexity while maintaining the overall performance (such as total travel time) of the traffic network.

Development of Control Techniques for AC Microgrids: A Critical Assessment

This article aims to provide a comprehensive review of control strategies for AC microgrids (MG) and presents a confidently designed hierarchical control approach divided into different levels. These levels are specifically designed to perform functions based on the MG’s mode of operation, such as grid-connected or islanded mode. The primary control ensures frequency (f) and voltage (V) stability, whereas the secondary control adjusts their values to their references and the tertiary control efficiently manages the power of distributed generators (DGs) in a cost-effective manner. The article extensively discusses centralized, decentralized, and distributed strategies for each control level, highlighting their differences, advantages, disadvantages, and areas of application. Finally, the usefulness of different control strategies at different levels is demonstrated through practical examples.

A hierarchical control approach to improve the voltage and frequency stability for hybrid microgrids-based distributed energy resources

This study proposes a multi-layer interactive control scheme to improve the performance of microgrids (MGs) based on distributed energy resources (DERs). This structure ensures the stability of the four system parameters, including voltage/frequency (V/F) and active/reactive (P/Q) power of the MGs, respectively, due to the correct operation of the defined control layers. In the proposed method, in the first layer, an internal voltage and current controller loop combined in the power droop controller (PDC), duties to adjust the unit V/F, in addition, the MG able to quickly respond to load oscillations is possible. Further, the secondary distributed V/F control strategies based on distributed finite-time control (DFTC) are implemented to modify and restore the control system. In this study, one of the novelties of the proposed method is the use of economically distributed modelling (EDM) in the upper layer above the other layers. The simulation results prove the efficiency of the proposed method strategy (hierarchical control) from two points of view, including voltage and frequency control; as the amplitude of fluctuations of voltage and frequency components has decreased for the units by 0.005 pu to 0.08 pu and 0.005 Hz to 0.018 Hz, respectively.

Multi-flow optimization of a greenhouse system: A hierarchical control approach

Greenhouses are implemented all over the world to increase agricultural production thanks to controlled environmental conditions (inside temperature, moisture and CO2 contents). However, such systems are energy-intensive. The presented work focuses on controlling a greenhouse’ onsite multi-energy system (gas, heat and electricity), extended to a multi-flow system as the CO2 produced by the energy units is used as a plant fertilizers. In this view, a three levels hierarchical control has been developed: a steady state economic MPC is combined with a dynamic Multi-energy MPC and low-level PID controllers. This new controller aims at determining the best synergies for economic flows management, subject to compliance with the required climatic conditions in the greenhouse. The proposed controller is applied to a greenhouse which energy system is based on a thermal energy storage fueled by a gas boiler and a combined heat and power unit. The results are confronted to a usual ruled-based controller performed by greenhouses owners, showing a more efficient dynamic functioning of the energy system. Consequently, less gas is consumed (80.79 t to 76.03 t), heat is produced when needed, and electricity from the CHP is economically optimized allowing less importation from the grid (from 17.80 MWh to 15.88 MWh), and profitable selling.

Finite-Time and Fixed-Time Consensus Tracking of Multiple Euler–Lagrange Systems via Hierarchical Control Approach

Finite-Time and Fixed-Time Consensus Tracking of Multiple Euler–Lagrange Systems via Hierarchical Control Approach

Distributed optimization of networked marine surface vehicles: A fixed-time estimator-based approach

This paper focuses on the fixed-time consensus problem of the networked marine surface vehicles (NMSVs) while minimizing the sum of the cost functions. To tackle this complex problem, we propose a fixed-time optimization hierarchical control approach that transforms the problem into two sub-problems: a distributed fixed-time optimization estimation problem and a local fixed-time tracking problem respectively. Specifically, the proposed approach leverages a distributed fixed-time optimization estimator to enable the NMSVs to estimate the state and velocity of a virtual leader while minimizing the sum of cost functions. Additionally, the above estimated signal will be tracked by the NMSVs under a fixed-time local tracking controller. We demonstrate the effectiveness of our proposed approach through simulation experiments on a network of four marine surface vehicles, and present the sufficient conditions for stability through Lyapunov stability analysis.

Long-term experimental study of price responsive predictive control in a real occupied single-family house with heat pump

The continuous introduction of renewable electricity and increased consumption through electrification of the transport and heating sector challenges grid stability. This study investigates load shifting through demand side management as a solution. We present a four-month experimental study of a low-complexity, hierarchical Model Predictive Control approach for demand side management in a near-zero emission occupied single-family house in Denmark. The control algorithm uses a price signal, weather forecast, a single-zone building model, and a non-linear heat pump efficiency model to generate a space-heating schedule. The weather-compensated, commercial heat pump is made to act smart grid-ready through outdoor temperature input override to enable heat boosting and forced stops to accommodate the heating schedule. The cost reduction from the controller ranged from 2-17% depending on the chosen comfort level. The experiment demonstrates that load shifting is feasible and cost-effective, even without energy storage, and that the current price scheme provides an incentive for Danish end-consumers to shift heating loads. However, issues related to controlling the heat pump through input-manipulation were identified, and the authors propose a more promising path forward involving coordination with manufacturers and regulators to make commercial heat pumps truly smart grid-ready.

A Review of Distributed Secondary Control Architectures in Islanded-Inverter-Based Microgrids

The increasing energy demand, the shortage of energy resources, and the environmental challenges faced by conventional power-generation systems are some of the ongoing challenges faced by modern power systems. Therefore, many efforts have been made by the scientific community to develop comprehensive solutions to overcome these issues. For instance, current technological advances have allowed the integration of distributed generators into the power systems, promoting the use of microgrids to overcome these issues. However, the use of renewable distributed generators have introduced new challenges to the traditional control system schemes. To overcome these challenges, a hierarchical control approach has been proposed for distributed renewable sources. In other words, the control scheme have been divided into three hierarchical levels, primary, secondary, and tertiary, to overcome the new challenges present in modern power systems. Due to extensiveness of this topic, this overview is focused on secondary control systems, mainly for AC isolated microgrids. To improve the power quality of modern systems, several secondary control schemes have been proposed to overcome the well-known problem of frequency and voltage deviation. Some of these schemes have also introduced adequate active/reactive power sharing techniques to optimize the utilization of resources. Additionally, other secondary control schemes have also focused on reducing the communication load, to lower the network cost and adding robustness against communication problems. This article presents an insight of the different control techniques used to overcome power quality and communication problems. A comprehensive overview of distributed secondary control techniques for islanded microgrids is presented. In addition, the implementation of these techniques is explained in an orderly and sequential manner.

Optimizing Subway Train Operation With Hierarchical Adaptive Control Approach

Optimizing Subway Train Operation With Hierarchical Adaptive Control Approach

Hierarchical Frequency Control of Hybrid Power Plants Using Frequency Response Observer

With frequency stability being challenged in modern power systems, transmission system operators have been designing new mitigation measures, such as fast frequency response (FFR), to maintain operation security of power systems. To accommodate technical requirements of new frequency control services (FCSs), the corresponding control should be implemented at asset controllers to enable fast responses. However, control counteraction can arise between plant controllers and asset controllers during the provision of FCSs. In this paper, a novel hierarchical frequency control approach is proposed to allow hybrid power plants (HPPs) to provide three types of FCSs, namely FFR, frequency containment response (FCR) and frequency restoration response (FRR). To solve control counteraction issue, an innovative frequency response observer (FROB) is proposed. The FROB at plant controllers and the hybrid power plant controller (HPPC) accurately estimates frequency response initiated by asset controllers, and the obtained estimation is used for control compensation at plant controllers and the HPPC to avoid control counteraction. Design guidelines and robustness analysis of the FROB are then discussed. Case studies are implemented in a power system dynamic model in MATLAB/Simulink, and the results show that the proposed frequency control approach enables coordinated operation of multiple technology power plants, with robust performance achieved when there are system uncertainties in HPPs.

A Hierarchical Control of Supercapacitor and Microsources in Islanded DC Microgrids

This paper proposes a hierarchical control approach for SuperCapacitor (SC) energy storage and microsources in islanded DC microgrids. It takes into account microsources’ dynamic and steady-state limitations. The proposed approach relies on an architecture presented for the SC’s and microsources’ interfaces with the DC bus. In the proposed architecture, DC microsources interface with the DC bus through buck converters. Moreover, the SC interfaces with the DC bus through the bi-directional buck converter. The instantaneous loads currents are measured and employed in the secondary control to calculate the reference currents for buck converters. In the primary control, we run buck converters as current sources. Moreover, the bi-directional buck converter is run to regulate the DC bus voltage. The proposed approach is generic; meaning that it can be employed for any islanded DC microgrid. Without loss of generality, to prove its effectiveness, we utilized it for a Low Voltage DC (LVDC) microgrid in this paper. Analysis, design, simulations, and experimental verifications are presented for the LVDC microgrid. The experimental results confirm that, by employing the proposed approach, the DC bus voltage deviations from its reference voltage is 5% and it restores in less than <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1~sec$ </tex-math></inline-formula> .

Hierarchical Optimal Maneuver Planning and Trajectory Control at On-Ramps With Multiple Mainstream Lanes

Connected Automated Vehicles (CAVs) have the potential to improve traffic operations when they cooperatively maneuver in merging sections. State-of-the-art approaches in cooperative merging either build on heuristics solutions or prohibit mainline CAVs to change lane on multilane highways. This paper proposes a hierarchical cooperative merging control approach that ensures collision-free and traffic-efficient merging through the interaction of a maneuver planner and an operational trajectory controller. The planner predicts future vehicular trajectories, including acceleration trajectories and time instants when lane changes start, in a long horizon up to 50 seconds with a linear prediction model. It establishes the optimal dynamic vehicle sequence in each lane by minimizing predicted traffic disturbances that can propagate upstream and lead to traffic breakdown. During the process, mainline vehicles may change lane to facilitate the on-ramp merging, albeit with a higher ego cost. The operational controller follows the established instructions from the planner and regulates vehicular trajectories with model predictive control in a shorter horizon of 6 seconds. The performance of the designed hierarchical cooperative merging control approach was compared to a cooperative merging method utilizing widely used first-in-first-out rule to establish merging sequences and the same operational controller to generate vehicular trajectories. Systematic comparison shows that the proposed approach consistently results in less disturbances during merging under 528 different scenarios with different traffic states, initial vehicular states, and desired time gap settings. On average, a decrease of 39.18% in disturbances was observed.

Data-Driven Strategies for Hierarchical Predictive Control in Unknown Environments

This article proposes a hierarchical learning architecture for safe data-driven control in unknown environments. We consider a constrained nonlinear dynamical system and assume the availability of state-input trajectories solving control tasks in different environments. In addition to task-invariant system state and input constraints, a parameterized environment model generates task-specific state constraints, which are satisfied by the stored trajectories. Our goal is to use these trajectories to find a safe and high-performing policy for a new task in a new, unknown environment. We propose using the stored data to learn generalizable control strategies. At each time step, based on a local forecast of the new task environment, the learned strategy consists of a target region in the state space and input constraints to guide the system evolution to the target region. These target regions are used as terminal sets by a low-level model predictive controller. We show how to <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i)</i> design the target sets from past data and then <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ii)</i> incorporate them into a model predictive control scheme with shifting horizon that ensures safety of the closed-loop system when performing the new task. We prove the feasibility of the resulting control policy, and apply the proposed method to robotic path planning, racing, and computer game applications. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This paper was motivated by the challenge of designing safe controllers for autonomous systems navigating through new environments. We consider scenarios where trajectory data from control tasks in different environments is available to the control designer. Possible applications include autonomous vehicles racing on new tracks or robotic manipulators performing tasks in the presence of new obstacles. Existing approaches to model-based control design for new environments generally use trajectory libraries, systematically adapting stored trajectories to the constraints of the new environment. This typically requires a priori knowledge of the entire task environment as well as resources to store and maintain the growing library. This paper suggests a new hierarchical control approach, in which stored trajectories are used to learn high-level strategies that can be applied while solving the new task. The strategies are learned offline, and only the parameterized strategy function needs to be stored for online control. Strategies only require knowledge of the nearby task environment, and provide navigation guidelines for the system. In this paper we show how to find such strategies from previous task data and how to integrate them into a low-level controller to safely and efficiently solve the new task. We also show how to adapt the modular framework as needed for a user’s desired application. Simulation experiments in robotic manipulator, autonomous vehicle, and computer game examples suggest that our approach can be used in a wide range of applications. In future research, we will address how to adapt the method for time-varying or stochastic environments.