Hierarchical Cooperative Research Articles

Distributed MPC-Based Hierarchical Cooperative Control for Mixed Vehicle Groups With T-CPS in the Vicinity of Traffic Signal Light

Distributed MPC-Based Hierarchical Cooperative Control for Mixed Vehicle Groups With T-CPS in the Vicinity of Traffic Signal Light

Sequencing-Enabled Hierarchical Cooperative CAV On-Ramp Merging Control With Enhanced Stability and Feasibility

Sequencing-Enabled Hierarchical Cooperative CAV On-Ramp Merging Control With Enhanced Stability and Feasibility

HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanism

Recently, some challenging tasks in multi-agent systems have been solved by some hierarchical reinforcement learning methods. Inspired by the intra-level and inter-level coordination in the human nervous system, we propose a novel value decomposition framework HAVEN based on hierarchical reinforcement learning for fully cooperative multi-agent problems. To address the instability arising from the concurrent optimization of policies between various levels and agents, we introduce the dual coordination mechanism of inter-level and inter-agent strategies by designing reward functions in a two-level hierarchy. HAVEN does not require domain knowledge and pre-training, and can be applied to any value decomposition variant. Our method achieves desirable results on different decentralized partially observable Markov decision process domains and outperforms other popular multi-agent hierarchical reinforcement learning algorithms.

Hierarchical Cooperative LSTM-Based Spectrum Sensing

In this letter, we design a hierarchical cooperative long short-term memory (LSTM) network-based cooperative spectrum sensing (CSS) method which utilizes convolutional neural network (CNN) and LSTM network. The CNN extracts spatial features from the input covariance matrices (CMs) which are generated by sensing data of each secondary user (SU) and the sequence of spatial features corresponding to multiple sensing periods are fed into secondary user LSTM (SU-LSTM) so that the PU activity pattern at SU level can be learned. The cooperative LSTM learns the group-level PU activity pattern from all SU-level temporal feature representations. The aim of learning the PU activity pattern at SU-level and group-level is to improve the detection performance further. To demonstrate the robustness of the proposed model, the scenario of an imperfect reporting channel is taken into account. With a sufficient amount of simulations, the effectiveness of the proposed method is proven and simulation results demonstrate that the proposed method outperforms the state-of-the-art in terms of detection probability and classification accuracy.

A Hierarchical Cooperative Frequency Regulation Control Strategy of Wind-Storage-Load in a Microgrid Based on Model Prediction

In order to give full play to the frequency regulation ability of multiple types of resources such as wind power, energy storage, and controllable load in a microgrid, this paper proposes a hierarchical cooperative frequency regulation control strategy of wind-storage-load in a microgrid based on model prediction. Firstly, according to the operation characteristics of each resource in the microgrid, a hierarchical cooperative frequency regulation architecture of wind-storage-load is constructed. On this basis, the frequency regulation control models of wind power, energy storage, and controllable load are established, respectively, and the calculation method of the characteristic index of the system frequency response is proposed. Then, taking the maximum frequency deviation as the stratification index, a hierarchical cooperative frequency regulation control strategy of wind-storage-load based on model prediction is proposed, and a power compensation strategy for connecting the wind turbine frequency support is proposed for the wind turbine speed recovery stage. Finally, a microgrid model including wind power, energy storage, and controllable load is built on Matlab/Simulink for simulation analysis. The simulation results show that the proposed control strategy can control wind power, energy storage, and controllable load to participate in frequency modulation in advance, and improve the frequency stability of the system.

Energy-Aware Coded Transmission Strategy for Hierarchical Cooperative Caching Networks

With the rapid development of millions more edge base stations (EBSs) and billions of user devices in the future networks, the desire for low-energy system design and transmission strategy will be even more compelling. In this letter, we investigate the energy consumption issue of hierarchical cooperative caching networks (HCCNs) in which both EBSs and users are capable of storing content data in their local caches. A three-stage coded transmission strategy in HCCNs is proposed, and the closed expression of energy consumption of each stage is derived. In addition, we formulate a total energy consumption minimization problem with the constraint of EBSs’ cache sizes, and provide an algorithm to obtain the optimal cache placement matrix. We demonstrate by numerical results that, our optimized three-stage coded transmission strategy can achieve lower energy consumption than various multicast transmission strategies under the different users’ cache sizes.

Internet-of-Things-augmented dynamic route planning approach to the airport baggage handling system

This paper proposes a dynamic route planning approach to the airport baggage handling systems. In previous studies, when planning the baggage transmission routes, real-time conditions of the baggage handling system are often neglected. This study incorporates the real-time baggage tracking information and device disruption information into the baggage route planning algorithm based on Internet-of-Things technologies. Taking Hierarchical Cooperative A* algorithm (HCA*) as the basic routing core, we propose a comprehensive algorithmic framework for the dynamic baggage route planning problem, in order to iteratively generate conflict-free baggage transmission routes while adapting to varying system conditions. We propose a unified prioritizing scheme so that baggage can be handled in proper order according to their arrival, departure and waiting time. This solution approach also contains a disruption handling mechanism by which the affected transportation links and baggage can be appropriately handled. A real airport case is used to verify the proposed solution approach. The results show that our proposed method is around 10% more efficient than the decentralized heuristic algorithm. Meanwhile, the average baggage throughput time decreases about 5% when real-time information is included. This paper also shows how to identify susceptible transmission links so specific maintenance plans can be arranged.

Hierarchical Cooperative Control Strategy for Battery Storage System in Islanded DC Microgrid

This paper presents a novel hierarchical cooperative control strategy to solve the problems of unbalanced State of Charge (SoC), unreasonable load current sharing, and unstable bus voltage for battery storage system (BSS) in islanded DC microgrid. In the communication layer, a neighbor-to-neighbor communication network architecture is constructed in which each battery storage unit (BSU) only communicates information with neighboring nodes, and a multi-agent consensus algorithm is used to iteratively estimate the relevant average global variables. In the main control layer, the droop coefficient is dynamically adjusted by constructing a fast SoC convergence function to ensure that the load current is reasonably distributed based on the rated capacity of each BSU. In the secondary control layer, a virtual voltage drop equalization controller and an average bus voltage compensation controller are designed to eliminate the impact of line impedance mismatch and compensate the average bus voltage. Furthermore, an optimized factor is designed to accelerate the later stage of SoC convergence. Finally, a DC microgrid experimental platform based on StarSim HIL is built, and the results demonstrate that the proposed strategy can achieve the control objectives of fast SoC balance, proportional load current sharing, and bus voltage stability.

Hierarchical Cooperative Control of Connected Vehicles: From Heterogeneous Parameters to Heterogeneous Structures

As one of the typical applications of connected vehicles (CVs), the vehicle platoon control technique has been proven to have the advantages of reducing emissions, improving traffic throughout and driving safety. In this paper, a unified hierarchical framework is designed for cooperative control of CVs with both heterogeneous model parameters and structures. By separating neighboring information interaction from local dynamics control, the proposed framework is designed to contain an upper-level observing layer and a lower-level tracking control layer, which helps address the heterogeneity in vehicle parameters and structures. Within the proposed framework, an observer is designed for following vehicles to observe the leading vehicle's states using neighboring communication, while a tracking controller is designed to track the observed leading vehicle using local feedback control. Closed-loop stability in the absence and presence of communication time delay is analyzed, and the observer is further extended to a finite time convergent one to address string stability under general communication topology. Numerical simulation and field experiment verify the effectiveness of the proposed method.

Learning to Cooperate: A Hierarchical Cooperative Dual Robot Arm Approach for Underactuated Pick-and-Placing

Cooperative multiagent manipulation systems allow to extend on the manipulative limitations of individual agents, increasing the complexity of the manipulation tasks the ensemble can handle. Controlling such a system requires meticulous planning of subsequent subtasks, queried to the individual agents, in order to execute the master task successfully. Real-time (re)planning is essential to ensure the task can still be achieved when subtasks execution suffers from uncertainty or when the master task changes intermittently requiring real-time reconfiguration of the plan. In this article, we develop a supervisory control architecture tailored to the cooperation of two robotic manipulators equipped with standard pick-and-place facilities in the plane. We control the planar position and orientation of an object using two underactuated manipulators so that only the position of the object can be controlled directly. The desired orientation follows from the accumulation of alternating relative angles. A time-invariant policy function is trained using deep reinforcement learning, which can determine a finite sequence of pick-and-place maneuvers to manipulate the object to a desired configuration. Two policy architectures are compared. The first uses the kinematic model to determine the final step, whilst the second policy makes this decision itself. The more information is given to the policy, the easier it trains. In return, it becomes less adaptable and loses some of its generalizability.

Cooperative Pathfinding

Cooperative Pathfinding is a multi-agent path planning problem where agents must find non-colliding routes to separate destinations, given full information about the routes of other agents. This paper presents three new algorithms for efficiently solving this problem, suitable for use in Real-Time Strategy games and other real-time environments. The algorithms are decoupled approaches that break down the problem into a series of single-agent searches. Cooperative A* (CA*) searches space-time for a non-colliding route. Hierarchical Cooperative A* (HCA*) uses an abstract heuristic to boost performance. Finally, Windowed Hierarchical Cooperative A* (WHCA*) limits the space-time search depth to a dynamic window, spreading computation over the duration of the route. The algorithms are applied to a series of challenging, maze-like environments, and compared to A* with Local Repair (the current video-games industry standard). The results show that the new algorithms, especially WHCA*, are robust and efficient solutions to the Cooperative Pathfinding problem, finding more successful routes and following better paths than Local Repair A*

Multiple Time-Varying Formation of Networked Heterogeneous Robotic Systems via Estimator-Based Hierarchical Cooperative Algorithms

This paper investigates both the time-varying formation and multiple time-varying formation tracking problems of networked heterogeneous robotic systems (NHRSs) with parameter uncertainties and external disturbances in the task space. Each robot inside can be either redundant or nonredundant. Several novel estimator-based hierarchical cooperative (EBHC) algorithms are designed to achieve both the tracking task and the possible preset subtasks for redundant robots. Besides, the designed estimator algorithms guarantee that each robot can obtain the accurate information of their corresponding leaders. By employing Lyapunov stability and input-to-state stability, sufficient conditions on the asymptotic stability of the error closed-loop system are derived. Finally, two simulation examples are presented to verify the effectiveness of the proposed algorithms.

Hierarchical Cooperation Improves Delay in Cognitive Radio Networks with Heterogeneous Mobile Secondary Nodes

This paper characterizes the throughput and delay performance of Cognitive Radio Networks (CRNs), where both primary and secondary networks coexist in a unit torus. Specifically, the primary network consists of static primary nodes (PNs) of density $n$ n , which have a higher priority to access the spectrum. In contrast, the secondary network consists of mobile secondary nodes (SNs) of density $m=n^{\beta }$ m = n β with $\beta \geq 1$ β ≥ 1 , which move according to a hybrid random walk mobility model and have opportunistic access to the spectrum without affecting primary packet transmissions. Motivated by the fact that cooperation between primary and secondary nodes leads to possible improvement on the performance of CRNs, as well as the fact that the heterogeneous moving regions of secondary nodes will bring about further improvement, we propose a novel hierarchical cooperative scheduling mechanism, where secondary nodes serve as relays for primary packet transmissions by exploiting their mobility heterogeneity and geographic information. Our findings include: (i) For the primary network, stronger mobility heterogeneity of secondary nodes leads to better delay performance of the primary network, and meanwhile the delay scaling can be significantly reduced to $\Theta (n^{\sqrt{\beta /{(4\;\log n)}}}\log ^{3/2}n)$ Θ ( n β / ( 4 log n ) log 3 / 2 n ) when a near-optimal per-node throughput of $\Theta ({1}/{\log n})$ Θ ( 1 / log n ) is obtained. (ii) For the secondary network, we also adopt a similar hierarchical cooperative scheduling mechanism, and obtain a near-optimal per-node throughput of $\Theta {(1/{\log m})}$ Θ ( 1 / log m ) with the delay scaling of $\Theta ({m^{1-(1/\sqrt{\log m})}})$ Θ ( m 1 - ( 1 / log m ) ) . (iii) The delay of secondary source-destination pairs is determined by the moving region of destinations and has no relation with sources. Our work provides deeper understandings of the cooperation, heterogeneous mobility, and geographic information on the performance of CRNs, and sheds light on designing more efficient CRNs.

Noncovalent Synthesis of Self-Assembled Nanotubes through Decoupled Hierarchical Cooperative Processes.

Because of their wide number of biological functions and potential applications, self-assembled nanotubes constitute highly relevant targets in noncovalent synthesis. Herein, we introduce a novel approach to produce supramolecular nanotubes with defined inner and outer diameters from rigid rod-like monomers programmed with complementary nucleobases through two distinct, decoupled cooperative processes of different hierarchy and acting in orthogonal directions: chelate cooperativity, responsible for the formation of robust Watson-Crick H-bonded cyclic tetramers, and nucleation-growth cooperative polymerization.

Game-Based Hierarchical Cooperative Control for Electric Vehicle Lateral Stability via Active Four-Wheel Steering and Direct Yaw-Moment Control

A Stackelberg game-based cooperative control strategy is proposed for enhancing the lateral stability of a four-wheel independently driving electric vehicle (FWID-EV). An upper‒lower double-layer hierarchical control structure is adopted for the design of a stability control strategy. The leader‒follower-based Stackelberg game theory (SGT) is introduced to model the interaction between two unequal active chassis control subsystems in the upper layer. In this model, the direct yaw-moment control (DYC) and the active four-wheel steering (AFWS) are treated as the leader and the follower, respectively, based on their natural characteristics. Then, in order to guarantee the efficiency and convergence of the proposed control strategy, a sequential quadratic programming (SQP) algorithm is employed to solve the task allocation problem among the distributed actuators in the lower layer. Also, a double-mode adaptive weight (DMAW)- adjusting mechanism is designed, considering the negative effect of DYC. The results of cosimulation with CarSim and Matlab/Simulink demonstrate that the proposed control strategy can effectively improve the lateral stability by properly coordinating the actions of AFWS and DYC.

A Hierarchical Cooperative Mission Planning Mechanism for Multiple Unmanned Aerial Vehicles

In this paper, the cooperative multi-task online mission planning for multiple Unmanned Aerial Vehicles (UAVs) is studied. Firstly, the dynamics of unmanned aerial vehicles and the mission planning problem are studied. Secondly, a hierarchical mechanism is proposed to deal with the complex multi-UAV multi-task mission planning problem. In the first stage, the flight paths of UAVs are generated by the Dubins curve and B-spline mixed method, which are defined as “CBC)” curves, where “C” stands for circular arc and “B” stands for B-spline segment. In the second stage, the task assignment problem is solved as multi-base multi-traveling salesman problem, in which the “CBC” flight paths are used to estimate the trajectory costs. In the third stage, the flight trajectories of UAVs are generated by using Gaussian pseudospectral method (GPM). Thirdly, to improve the computational efficiency, the continuous and differential initial trajectories are generated based on the “CBC” flight paths. Finally, numerical simulations are presented to demonstrate the proposed approach, the designed initial solution search algorithm is compared with existing methods. These results indicate that the proposed hierarchical mission planning method can produce satisfactory mission planning results efficiently.

Hierarchical Cooperative Discovery of Personal Places from Location Traces

It is becoming increasingly important to accurately detect a user's presence at certain locations during certain times of the day, e.g., to study the user's patterns with respect to mobility, behavior, or social interactions and to enable the delivery of targeted services. However, instead of geographic locations, it is often more important to determine a locale that is relevant to the user, e.g., the place of work, home, homes of family and friends, social gathering places, etc. These significant personal places can be determined through analysis, e.g., via segmentation of location traces into a discrete sequence of places. However, segmentation of traces with many gaps (e.g., due to loss of network connectivity or GPS signal) results in a large number of small segments, where many of these segments actually belong together. This work proposes a novel segmentation approach that opportunistically fills gaps in a user's location trace by borrowing location data from other co-located users utilizing the power of mobile crowd sensing and computing (MCSC) paradigm. Through our analysis of four separate large-scale crowd sensing study datasets, we show that our approach yields more and larger segments than the state-of-the-art, where each segment accurately represents the presence of a user at a significant personal place.

An efficient multi-core implementation of the Jaya optimisation algorithm

ABSTRACTIn this work, we propose a hybrid parallel Jaya optimisation algorithm for a multi-core environment with the aim of solving large-scale global optimisation problems. The proposed algorithm is called HHCPJaya, and combines the hyper-population approach with the hierarchical cooperation search mechanism. The HHCPJaya algorithm divides the population into many small subpopulations, each of which focuses on a distinct block of the original population dimensions. In the hyper-population approach, we increase the small subpopulations by assigning more than one subpopulation to each core, and each subpopulation evolves independently to enhance the explorative and exploitative nature of the population. We combine this hyper-population approach with the two-level hierarchical cooperative search scheme to find global solutions from all subpopulations. Furthermore, we incorporate an additional updating phase on the respective subpopulations based on global solutions, with the aim of further improving the convergence rate and the quality of solutions. Several experiments applying the proposed parallel algorithm in different settings prove that it demonstrates sufficient promise in terms of the quality of solutions and the convergence rate. Furthermore, a relatively small computational effort is required to solve complex and large-scale optimisation problems.

Evaluation of a Hierarchical Cooperative Transport System Using Demand Responsive Bus on a Dynamic Simulation

Evaluation of a Hierarchical Cooperative Transport System Using Demand Responsive Bus on a Dynamic Simulation

New Aspects for the Hierarchical Cooperative Motions in Photoalignment Process of Liquid Crystalline Block Copolymer Films

The realignment process of azobenzene-containing liquid crystalline (LC) block copolymer films by irradiation with linearly polarized light (LPL) was investigated in detail by time-resolved grazing incidence X-ray scattering measurements using a synchrotron beam. By using a newly synthesized diblock copolymer possessing a poly(hexyl methacrylate (PHMA) block with a lowered glass transition temperature (Tg = −5 °C), intense X-ray scattering signals were obtained, which favorably provided precise and detailed time course profiles and information on the interplay between the hierarchical structures involved along the realignment process. The LC layers were softened and fluctuated at an early stage by LPL irradiation which induced shrinkage of the cylinder to cylinder distance of microphase separation patterns before the alignment change. Interestingly, the ordering process of the MPS cylinder array of larger hierarchical structure proceeded faster than that of the azobenzene LC layer of smaller feature size in the realigned state. Decisive evidence was obtained to support the subdomain rotation mechanism in the realignment process.