Lower Layer Model Research Articles

An Independently Learnable Hierarchical Model for Bilateral Control-Based Imitation Learning Applications

Recently, motion generation by machine learning has been actively researched to automate various tasks. Imitation learning is one such method that learns motions from data collected in advance. However, executing long-term tasks remains challenging. Therefore, a novel framework for imitation learning is proposed to solve this problem. The proposed framework comprises upper and lower layers, where the upper layer model, whose timescale is long, and lower layer model, whose timescale is short, can be independently trained. In this model, the upper layer learns long-term task planning, and the lower layer learns motion primitives. The proposed method was experimentally compared to hierarchical RNN-based methods to validate its effectiveness. Consequently, the proposed method showed a success rate equal to or greater than that of conventional methods. In addition, the proposed method required less than 1/20 of the training time compared to conventional methods. Moreover, it succeeded in executing unlearned tasks by reusing the trained lower layer.

Bi-layer optimal planning of rural distribution network based on KKT condition and Big-M method

With the increasing seriousness of environmental and energy shortage problems, new technologies such as photovoltaic distributed generations and energy storage devices will be the development trend of future rural distribution network. In this paper, a bi-layer optimal planning model of rural distribution network is established, where the optimal planning problem on the long time scale is solved in the upper-layer model, and the optimal operation problem on the short time scale is solved in the lower-layer model. Then, the lower-layer optimal operating model is converted and added into the constraints of the upper-layer model according to the Karush–Kuhn–Tucker (KKT) conditions, and the Big-M method is used to linearize the nonlinear problems in the constraints of the proposed model. Finally, the case study of a rural distribution network in Zhejiang, China is demonstrated for verifying the effectiveness and advantages of the proposed bi-layer model on alleviating the pressure of power supply and increase profits from electricity sales.

A two-layer non-hydrostatic landslide model for tsunami generation on irregular bathymetry. 1. Theoretical basis

We describe a two-layer, coupled model for landslide motion over irregular bathymetry and resulting water column motion and tsunami generation. The three-dimensional non-hydrostatic wave model NHWAVE (Ma et al., 2012) is applied as the upper-layer model to simulate landslide-generated tsunami waves. Here, we focus on the derivation of governing equations for the lower-layer model, where the landslide is described as either a viscous mud flow or a saturated granular debris flow. The governing equations are depth-integrated in a Cartesian coordinate system referenced to the still water surface. Vertical acceleration of the slide is taken into account by including non-hydrostatic pressure effects within the slide. The model is validated in comparison to analytic results of Mangeney et al. (2000) and the slope-oriented numerical approach of Ma et al. (2015). The numerical approach and further examples are described in the companion paper Zhang et al. (2021).

Two-layered dynamic control for simultaneous set-point tracking and improved economic performance

This work introduces a multi-objective optimization strategy to handle conflicting set-point tracking and economic objectives in a two-layer hierarchical control framework. A dynamic multi-objective real-time optimizer (DMO), incorporated in the upper layer, handles multiple control objectives with set-point tracking being the higher priority objective and computes optimal plant trajectories. This plant-wide trajectory information is communicated to the lower-layer model predictive control (MPC) operating at a faster sampling rate. The conventional weight-based and lexicographical method for the DMO are discussed. A new algorithm is conceptualized based on the lexicographical method to handle prioritized objectives. The proposed algorithm modifies the higher priority tracking objective and establishes improved economic performance compared to the conventional techniques, with minimal effect on the conflicting tracking objective, through a systematic choice of the preferred Pareto solution. The proposed algorithm’s efficacy, within the hierarchical framework, is analyzed using two case studies: A polymerization reactor and a multi-unit reactor–separator system.

Robust purchase and sale transactions optimization strategy for electricity retailers with energy storage system considering two-stage demand response

A new two-stage demand response is designed for the electricity retailers with energy storage system (ESS-ER) in the deregulated power market. The ESS-ER could response to the output of different power sources by adjusting the charging-discharging behavior according to the bidding power price. The paper models the two-stage demand response for electric power retailers and proposed a two-layer coordinated optimal model for the purchase and sale of the electric power retailers. In the upper layer model, the conditional value at risk method and robust stochastic theory are applied to describe the uncertainty influence of wind power and Photovoltaic (PV) power, and the minimum whole cost of power purchasing is taken as the objective. In the lower-layer, the power consumption behaviors of different customers are considered to get the maximum revenue of power selling by implementing differentiated demand response. Then, to solve the two-layer mathematical model, the lower-layer model is converted into the Karush-Kuhn-Tucker (KKT) optimality conditions. The results show that: (1) The two-stage demand response could smooth the curves of power purchasing and terminal users’ load, which could bring more flexible transaction space. (2) The proposed two-layer transaction model could balance the cost and risk of power purchasing, bringing more trading opportunities for wind power and PV, which can also reduce the energy consumption cost of the end-users. (3) By introducing the risk cost coefficient, confidence degree and robust coefficient, the decision-makers can adjust the power trading behaviors, and establish the optimal power trading scheme in line with their expected situation. (4) When higher energy storage capacity is set, the efficiency of demand response rises. When the capacity ratio of wind to energy storage is 4:1, the efficiency of demand response reaches the best. When larger energy storage capacity is set, the demand response turns to be more effective. However, when the capacity ratio of wind and PV to energy storage is 4:1, the effect of demand response reaches the best. Overall, the proposed model could provide an effective tool for power retailers in China's electric power market.

A two-stage optimal coordinated scheduling strategy for micro energy grid integrating intermittent renewable energy sources considering multi-energy flexible conversion

In this paper, a two-stage coordinated scheduling model for dealing with the optimization problem of multiple-energy (power, heating, and cooling) synergistic supply in the micro energy grid (MEG) is proposed. To overcome the uncertainty impact of wind power and photovoltaic power, the scheduling cycle was divided into a day-ahead phase and a real-time phase. Then, the day-ahead forecasting results were taken as random variables for the upper-layer model. The actual value was taken as the realization of random variables for the lower-layer model, including the energy storage revised model and demand response (DR) scheduling model. The cell membrane and chaotic search algorithms were employed to improve the particle swarm algorithm. The “international low carbon park” in China was selected for a case study. The case study showed the following results: (1)The two-stage optimization model and solution algorithm achieved the optimal synergistic supply of multiple energy forms. (2) Power-to-gas could convert the surplus electricity into natural gas, realizing multi-directional energy cascade conversion. (3)MEG could flexibly interact with the upper-grade energy network for obtaining more economic benefits. (4)The price-based DR could smooth the energy load curve and maximize the operation revenue of the MEG by utilizing complementary characteristics of energy prices.

Research on Micro-Grid Group Intelligent Decision Mechanism under the Mode of Block-Chain and Multi-Agent Fusion

This paper aims to study the problems of surplus interaction, poor real-time performance, and excessive processing of information in the micro-grid scheduling and decision-making process. Firstly, the micro-grid dual-loop mobile topology structure is designed by using the method of block-chain and multi-agent fusion, realizing the real-time update of the decision-making body. Secondly, on the basis of optimizing the decision-making body, a two-layer model of intelligent decision-making under the decentralized mechanism is established. Aiming at the upper model, based on the theory of block-chain consensus mechanism, this paper proposes an improved evolutionary game algorithm. The maximum risk-benefit in the decision-making process is the objective function, which realizes the evaluation and optimization of decision tasks. For the lower layer model, based on the block-chain distributed ledger theory, this paper proposes an improved hybrid game reinforcement learning algorithm, with the maximum controllable load participation as the objective function, and realizes the optimal configuration of distributed energy in the micro-grid. This paper reveals the rules of group intelligent decision making in micro-grid under multi-task. Finally, the effectiveness of the proposed algorithm is verified by using Beijing Jin-feng Energy Internet Park data.

Research on optimal control method of distributed generation considering the influence of controllable load

On the basis of optimal allocation of distributed generation in conventional distribution network, and considering the characteristics of active distribution system and the influence of controllable loads, a two-layer optimal configuration model of distributed generation is established. The upper layer model is used to solve the optimal allocation and sizing of distributed power sources for active distribution systems and the lower layer model is used to solve the optimal strategy of controllable loads in each period. The output uncertainty of uncontrollable distributed generators, such as wind power generation and photovoltaic power generation, is characterized by box uncertain set based on robust optimization theory.With this model, the optimal configuration of distributed generation can be solved and the factors of operation and dispatch can be taken into account. To some extent, the overall planning and operation of active distribution system can be realized.Finally, the genetic algorithm is used to solve the two-layer optimization model.The rationality of the proposed model, the applicability of genetic algorithm, and the strong ability of global optimization are verified by case studies of IEEE 33-bus distribution system.

A bi-level stochastic scheduling optimization model for a virtual power plant connected to a wind–photovoltaic–energy storage system considering the uncertainty and demand response

To reduce the uncertain influence of wind power and solar photovoltaic power on virtual power plant (VPP) operation, robust optimization theory (ROT) is introduced to build a stochastic scheduling model for VPP considering the uncertainty, price-based demand response (PBDR) and incentive-based demand response (IBDR). First, the VPP components are described including the wind power plant (WPP), photovoltaic generators (PV), convention gas turbine (CGT), energy storage systems (ESSs) and demand resource providers (DRPs). Then, a scenario generation and reduction frame is proposed for analyzing and simulating output stochastics based on the interval method and the Kantorovich distance. Second, a bi-level robust scheduling model is proposed with a double robust coefficient for WPP and PV. In the upper layer model, the maximum VPP operation income is taken as the optimization objective for building the scheduling model with the day-ahead prediction output of WPP and PV. In the lower layer model, the day-ahead scheduling scheme is revised with the actual output of the WPP and PV under the objectives of the minimum system net load and the minimum system operation cost. Finally, the independent micro-grid in a coastal island in eastern China is used for the simulation analysis. The results illustrate that the model can overcome the influence of uncertainty on VPP operations and reduce the system power shortage cost by connecting the day-ahead scheduling with the real-time scheduling. ROT could provide a flexible decision tool for decision makers, effectively addressing system uncertainties. ESSs could replace CGT to provide backup service for the WPP and PV, to smooth the VPP output curve and to improve the WPP and PV grid connection by its charging–discharging characteristics. Meanwhile, IBDR and PBDR could smooth the load curve to the maximum extent, link the generation side with the demand side to minimize abandoned power value and reach the optimum benefit of VPP operation.

A Bi-Level Energy-Saving Dispatch in Smart Grid Considering Interaction Between Generation and Load

The energy-saving dispatch could significantly enhance the energy consumption and carbon dioxide emission reduction, as well as the sustainable development of the socio economy in China. With the rapid growth of smart grid, the use of demand response to dispatch loads with flexible consumption time and/or quantity has been a new trend in power industry. This paper proposes a new energy-saving dispatch problem while considering energy-saving and emission-reduction potentials of generation and demand sides, as well as the interaction between the two. A bi-level optimization model is established to address the interaction between the energy-saving dispatch of thermal units and that of users. The objective function of the upper level considers both the electric power generation cost and the carbon emission cost of thermal units, while the lower level integrates both compensation and incentive costs of electricity consumers into its objective function according to the influences of their regulation-based demand response to the power grid. Moreover, user benefits of reducing downtime and avoiding frequent load restarting are also considered in the lower layer model. An iterative algorithm is proposed and the improved nondominated sorting genetic algorithm II (NSGA-II) method is used to solve the lower-layer model for seeking the optimal compromise solution on the Pareto frontier, which is derived by maximum deviations and entropy-based multiple attributes decision making method. Comparing with general NSGA-II and multiobjective genetic algorithms, the improved NSGA-II method can improve the spatial distribution of Pareto solution set and reduce the number of iterations, thus having stronger consistency among multiple objective functions and better performance.

A two-layer queueing model to predict performance of packet transfer in broadband networks

In this paper, we propose an approximate analytical method to predict the performance of packet transfer in broadband networks. Since the ATM is a promising way to achieve a multimedia data transmission in broadband networks, the underlying network is assumed to be an ATM network. To represent the packet-level and the cell-level data transfers, we introduce a two-layer queueing network model. In the higher layer of the model, the source node (the sender) generates packets and then segments the packets into cells, while the destination node (the receiver) reassembles the arriving cells into the packets. The lower layer model is a tandem queueing network in which each queue corresponds to an ATM switch. We employ a discrete-time analysis of the queueing network model to obtain the packet-level performance measures, such as the end-to-end packet transfer delay, the packet loss probability, and the throughput. In numerical examples, we show some packet transfer delay distributions and effects of the packet size on the packet loss probability.