Multi-timescale Control Research Articles

Multitimescale Control and Communications With Deep Reinforcement Learning—Part I: Communication-Aware Vehicle Control

Multitimescale Control and Communications With Deep Reinforcement Learning—Part I: Communication-Aware Vehicle Control

Multitimescale Control and Communications With Deep Reinforcement Learning—Part II: Control-Aware Radio Resource Allocation

Multitimescale Control and Communications With Deep Reinforcement Learning—Part II: Control-Aware Radio Resource Allocation

Event-triggered multi-time scale control and low carbon operation for electric-hydrogen DC microgrid

This paper addresses the escalating need for low-carbon solutions and discusses the integrated control system for the electric-hydrogen DC microgrids. A multi-time scale control mechanism for the DC microgrid is proposed, which aims to enhance its adaptability and scalability. The proposed multi-time control scheme comprises two control layers. The system-level control layer utilizes optimization to address economic issues and carbon trading, aiming to minimize operational costs and carbon emissions in the microgrid. The local-level control layer incorporates an advanced nonlinear control method based on the exponential reaching law-powered terminal sliding mode control (ERL-TSMC) for accurate tracking capabilities at the local-level and to handle dynamic control challenges in real-time. Furthermore, an event-triggered control mechanism is introduced to reduce considerable control efforts while guaranteeing system robustness under various disturbances. Both simulations and hardware experiments demonstrate that the proposed control method guarantees efficient low-carbon operation, achieving a significantly lower average carbon intensity of 0.19 kg CO2/kWh. Additionally, it exhibits robust performance at the local-level, with a steady-state error of only 0.03%.

A contraction-constrained economic Model Predictive Control for nonlinear processes using multi-timescale models

Many chemical processes exhibit diverse timescale dynamics with a strong coupling between timescale sensitive variables. Model predictive control with a non-uniformly spaced optimisation horizon is an effective approach to multi-timescale control and offers opportunities for reduced computational complexity. In such an approach, dynamics with different timescales can be included in an optimisation problem by implementing faster timescale models (with shorter time intervals) earlier in the prediction horizon and increasingly slower timescale models (with longer time intervals) towards the end. In this paper, a reference-flexible condition is developed based on contraction theory to provide a stability guarantee for cost minimising control of multiple timescale nonlinear systems using multiple predictive models.

Research on the multi-timescale optimal voltage control method for distribution network based on a DQN-DDPG algorithm

A large number of distributed generators (GDs) such as photovoltaic panels (PVs) and energy storage (ES) systems are connected to distribution networks (DNs), and these high permeability GDs can cause voltage over-limit problems. Utilizing new developments in deep reinforcement learning, this paper proposes a multi-timescale control method for maintaining optimal voltage of a DN based on a DQN-DDPG algorithm. Here, we first analyzed the output characteristics of the devices with voltage regulation function in the DN and then used the deep Q network (DQN) algorithm to optimize the voltage regulation over longer times and the deep deterministic policy gradient (DDPG) algorithm to optimize the voltage regulation mode over short time periods. Second, the design strategy of the DQN-DDPG algorithm as based on the Markov decision process transformation was presented for the stated objectives and constraints considering the state of ES charge for prolonging the energy storage capacity. Lastly, the proposed strategy was verified on a simulation platform, and the results obtained were compared to those from a particle swarm optimization algorithm, demonstrating the method’s effectiveness.

Multi-time scale control and optimization via averaging and singular perturbation theory: From ODEs to hybrid dynamical systems

Multi-time scale techniques based on singular perturbations and averaging theory are among the most powerful tools developed for the synthesis and analysis of feedback control algorithms. This paper introduces some of the recent advances in singular perturbation theory and averaging theory for continuous-time dynamical systems modeled as ordinary differential equations (ODEs), as well as for hybrid dynamical systems that combine continuous-time dynamics and discrete-time dynamics. Novel multi-time scale analytical tools based on higher-order averaging and singular perturbation theory are also discussed and illustrated via different examples. In the context of hybrid dynamical systems, a class of sufficient Lyapunov-based conditions for global stability results are also presented. The analytical tools are illustrated through various new architectures and algorithms within the context of adaptive and extremum-seeking systems. These tools are suitable for the study of model-free optimization and stabilization problems that require the synergistic use of continuous-time and discrete-time feedback. The paper aims to acquaint the reader with a range of modern tools for studying multi-time scale phenomena in optimization and control systems, providing some guidelines for future research in this field.

Small-signal synchronization stability of grid-forming converter influenced by multi time-scale control interaction

The grid-forming (GFM) converters that simulate the dynamics of synchronous generators (SGs) and actively participate in grid voltage and frequency regulation are increasingly employed as grid interfaces for renewable energy generation. Its synchronization stability with AC grid is also increasingly concerned in recent years. In this paper, the small-signal synchronization stability of GFM converters with virtual synchronous generator (VSG) control is investigated, with the influence of multi time-scale control interaction considered especially. First, based on multi time-scale dynamics analysis, an analogized Heffron–Phillips model is developed to physically depict the small-signal synchronization dynamics of VSG. The multi time-scale control interaction introduces an additional input torque to the equivalent motion of VSG’s synchronization behavior. Based on damping torque analysis, the synchronization stability is depended by the inherent damping component of VSG’s virtual rotor motion and the additional damping component introduced by multi time-scale control interaction. Qualitative and quantitative analysis reveal that the additional damping is negative and deteriorates the synchronization stability. When the negative additional damping is greater than the positive inherent damping, small-signal synchronization instability occurs due to insufficient damping. Finally, the influence factors including control parameters and grid strength are discussed.

Multi-time scale control of Southern Ocean diapycnal mixing over Atlantic tracer budgets

Oceanic cross-density (diapycnal) mixing helps sustain the ocean density stratification and its Meridional Overturning Circulation (MOC) and is key to global tracer distributions. The Southern Ocean (SO) is a key region where different overturning cells connect, allowing nutrient and carbon rich Indian and Pacific deep waters, and oxygen rich Atlantic deep waters to resurface. The SO is also rife with intense diapycnal mixing due to the interaction of energetic eddies and currents with rough topography. SO diapycnal mixing is believed to be of secondary importance for the MOC. Here we show that changes to SO mixing can cause significant alterations to biogeochemical tracer distributions over short and long time scales in an idealized model of the AMOC (Atlantic MOC). While such alterations are dominated by the direct impact of changes in diapycnal mixing on tracer fluxes on annual to decadal time scales, on centennial time scales they are dominated by the mixing-induced variations in the advective transport of the tracers by the AMOC. This work suggests that an accurate representation of spatio-temporally variable local and non-local mixing processes in the SO is essential for climate models’ ability to (i) simulate the global biogeochemical cycles and air sea carbon fluxes on decadal time scales, (ii) represent the indirect impact of mixing-induced changes to AMOC on biogeochemical cycles on longer time scales.

Multi-Time scale control framework of hybrid energy storage system via robust control and virtual impedance

Aiming at the problems of the strong randomness and fluctuation of the output power in photovoltaic power generation system, a multi-time scale control framework of hybrid energy storage system is proposed. Firstly, the complementary characteristics of charge/discharge between lithium battery and supercapacitor is analyzed. Then, The models of lithium battery energy storage unit and supercapacitor energy storage unit are established, respectively. Considering the different time scales of the output power response of lithium battery and supercapacitor, a short-time scale control framework is designed for supercapacitor energy storage unit based on the robust control, so that the bus voltage of system can be stable when the output power of photovoltaic power generation system changes or the load changes rapidly. At the same time, a long-time scale control framework is designed for lithium battery energy storage unit based on the virtual impedance, so that the voltage of the supercapacitor can be maintained within set value when the output power of the photovoltaic power generation system is stable. Therefore, the multi-time scale control framework can not only realize the large current charging/discharging for lithium battery and the power tracking at low frequency, but also realize the small current charging/discharging for supercapacitor and the power tracking at high frequency. Finally, the effectiveness of the proposed framework is verified by the simulation platform of photovoltaic power generation system.

Research on cluster operation optimization of multi‐port energy routers

The penetration rate of distributed new energy and DC load increase in the distribution network improve the new energy consumption rate and the reliability of power supply. This paper proposes an AC/DC hybrid system structure with multi-port energy router (ER) cluster, and compares the advantages of this structure over the AC/DC hybrid microgrid with multiple conversion links. In order to realize the stable, economic, and reliable operation of this structure, the multi-layer and multi-time scale control method and communication mode are established. According to the non-linear characteristics of energy consumption of ER port, the optimization model of the cluster control layer and the control mode of each port in the cluster system are studied. By setting several examples, the rationality of the proposed ER cluster operation control mode and the optimization method and the effectiveness in improving the total energy efficiency of the ER cluster system are verified; the advantages of ER cluster operation in improving the operation energy efficiency of key devices in the distribution network are illustrated.

Multi-Timescale Control of Variable-Speed Wind Turbine for Inertia Provision

The increasing deployment of power converters has led to a significant reduction in the power system inertia and consequently resulted in frequency stability issues. To improve the robustness of the grid against frequency disturbances, it is becoming more expected in many countries that renewable energy generation, such as wind turbine power systems, should provide equivalent inertia support to the power system. This can be achieved through advanced control of power converters, in addition to adding extra energy storage devices, e.g., batteries. In wind turbine systems, although the ancillary service of inertia provision can be realized by coupling the rotor speed with the grid frequency, the rotor speed recovery process affects the inertia response if the controller is not properly designed or well-tuned. To address this issue, in this paper, we propose a multi-timescale control strategy for a doubly fed induction generator (DFIG) wind turbine system. Synthetic inertia control and speed recovery control are simultaneously incorporated into the controller of the rotor-side converter, whereas their dynamics are decoupled under different timescales to avoid control conflict. Extensive simulation results are provided, which validate the efficacy of the proposed inertia emulation scheme.

Decomposition-Coordination-Based Voltage Control for High Photovoltaic-Penetrated Distribution Networks under Cloud-Edge Collaborative Architecture

With the integration of high proportional photovoltaics (PVs) into distribution networks, the superposition of uncertain output power of PVs and stochastic load demand fluctuations have posed a serious challenge to voltage stability. In this paper, a multitimescale decomposition-coordination-based voltage control method is proposed under the cloud-edge cooperative architecture. The distribution network is firstly decomposed into several subnetworks. In each subnetwork, the edge computing device is equipped to realize the distributed control and provide external computing resources for the cloud center. Then, a multitimescale control scheme containing the interval dispatch stage and the real-time time-window stage is proposed. In the interval dispatch stage, a cloud-edge collaborative calculation strategy considering balanced resource allocation is well designed to obtain the global optimal power flows and the corresponding reference operating points of the voltage control equipment. Meanwhile, in the real-time time-window stage, a consensus-based voltage correction mechanism under the optimal power flow boundary constraints is designed to avoid the voltage violations caused by unexcepted power fluctuations deviating from the representative scenarios. Simulation results with the improved 33-bus and IEEE 123-bus systems have demonstrated the effectiveness of our proposed method.

Power Management of AC/DC Hybrid Distribution Network With Multi-Port PET Considering Reliability of Power Supply System

In the current distribution network, photovoltaic, wind power, energy storage, and other distributed energy are widely connected, and the proportion of generalized DC load is rapidly increasing. With the development of power electronics technology, using multi-port power electronic transformer (PET) to achieve high-efficiency access of large-capacity AC/DC source and load is the current research hotspot, and AC / DC hybrid power supply is inevitable. The introduction of a large number of power electronics and the flexible coordinated control and complementary fault-tolerant advantages of PET bring challenges to the operation and maintenance management of the AC/DC hybrid power system. In this paper, the structure of the AC/DC hybrid power system with the multi-port PET cluster is introduced. Based on the idea of hierarchical and partitioned control, a three-layer and multi-time scale control mode of integrated automation system, PET cluster and PET controller is proposed; Aiming at the problem of reliability evaluation, a sequential Monte Carlo simulation method is proposed to simulate the AC/DC hybrid power system with the multi-port PET cluster. The influence of multi-port PET parallel cluster mode and port capacity limitation on the reliability of the power system is analyzed, and the effectiveness of the model and algorithm is verified. Finally, an example is given to verify that the proposed structure is conducive to improving the utilization level of renewable energy in the distribution network.

Two Level Control Strategy for Independent AC/DC Hybrid Microgrid Based on Integrated Control

Aiming at the independent AC/DC microgrid, a simple and effective multi time scale control strategy is proposed by adopting the combination of day ahead scheduling and real-time control. The day ahead scheduling strategy takes the minimum operation cost of AC/DC microgrid as the goal, and schedules the controllable loads in the AC/DC microgrid with particle swarm optimization, which makes full use of renewable energy. The real-time control strategy controls the output of the distributed generation, the load reduction and the switching of the diesel generator according to the real-time energy storage state of charge (SOC), which keep the power balance of the AC/DC microgrid. The effectiveness of the proposed two-level control strategy is verified by a practical AC/DC hybrid microgrid. The results show that the proposed method can make full use of renewable energy and ensure the economic and reliable operation of AC/DC hybrid microgrid.

Dual-Rate Adaptive Optimal Tracking Control for Dense Medium Separation Process Using Neural Networks

Dense medium separation (DMS) is of great significance for coal cleaning. The DMS control system always involves dense medium density adjustment and ash content control that are operating on fast and slow time scales, respectively. The inherent time-varying and strongly nonlinear characteristics of the DMS process give rise to challenges for the design of this multitime scale control system. To address this issue, this article proposes a dual-rate adaptive optimal tracking control approach for the DMS system. For the basic loop process, a nonlinear adaptive PI controller containing a neural network (NN)-based unmodeled dynamics compensator is proposed. Then, a lifting technique is used to unify the time scales of the two loops accompanied by formulating a generalized controlled object, whose dynamics is completely unknown. On this basis, a data-driven operation optimization control method that combines adaptive dynamic programming algorithm and reference control is developed, which is implemented using NNs. Finally, the stability of the proposed method is analyzed. The simulation results indicate its effectiveness.

Multi-Time Scale Coordinated Control and Scheduling of Inverter-Based TCLs With Variable Wind Generation

To address microgrid tie flow errors caused by wind generation variability, this paper proposes and develops a multi-time scale coordinated control and scheduling strategy for inverter-based thermostatically controlled loads (TCLs). First, in hour-time scale, inverter-based TCLs with adjusting temperature set-point are modeled as virtual generators to compensate tie flow deviations in the day-ahead plan. Next, in minute-time scale, virtual batteries representing operating behaviors of inverter -based TCLs with frequency control are scheduled determined by the control of virtual generators in hour-time scale. The virtual batteries are scheduled to smooth out tie flow errors corresponding to day-ahead plan and hour-time scale schedules. The multi-time scale control methods are coordinated to employ the response potential of inverter-based TCLs and response curve-based methods are proposed to control inverter-based TCLs considering the customer privacy. The multi-time scale stochastic schedules which are based on response curves of inverter-based TCLs are coordinated to accommodate wind generation variability. Simulation results demonstrate that the microgrid tie flow errors are effectively mitigated by the proposed multi-time scale coordinated control and scheduling of inverter-based TCLs.

Multi-time Scale Control of Dual-Stage Nanopositioning Systems

In this paper, a novel multi-time scale control technique is applied to a serial dual-stage nanopositioning system. Dual-stage nanopositioning systems combine a high-speed, short-range actuator and a low-speed, long-range actuator to achieve long-range and high-speed positioning. This results in a system that has relatively complicated dynamics due to the physical interaction between the two actuators and their different time-scales. In addition, models of these actuators can be ill-conditioned, which can lead to issues with numerical simulations and controller design. These issues make dual-stage nanopositioning systems well suited to multi-time scale control algorithms. In the proposed algorithm, the system is split (decoupled) into a set of subsystems, where each subsystem has an individual time scale and is independently controlled via state feedback. This alleviates the issues associated with ill-conditioning and simplifies controller design. This paper introduces the novel multi-time scale control design concept and its application to single-axis dual-stage nanopositioners - although it can be easily expanded to more complex systems, e.g., multi-axes, nanopositioning devices. The proposed control technique is validated through simulations of an experimentally obtained serially coupled dual-stage nanopositioning model.

Full-Body Optimal Control Toward Versatile and Agile Behaviors in a Humanoid Robot

In this letter, we develop an optimal control framework that takes the full-body dynamics of a humanoid robot into account. Employing full-body dynamics has been explored in, especially, an online optimal control approach known as model predictive control (MPC). However, whole-body motions cannot be updated in a short period of time due to MPC's large computational burden. Thus, MPC has generally been evaluated with a physical humanoid robot in a limited range of tasks where high-speed motion executions are unnecessary. To cope with this problem, our multi-timescale control framework drives whole-body motions with a computationally efficient hierarchical MPC. Meanwhile, a biologically inspired controller maintains the robot's posture for a very short control period. We evaluated our framework in skating tasks with simulated and real lower-body humanoids that have rollers on the feet. Our simulated robot generated various agile motions such as jumping over a bump and flipping down from a cliff in real time. Our real lower-body humanoid also successfully generated a movement down a slope.

A unified time scale intelligent control algorithm for micro grid based on extreme dynamic programming

Benefiting from the progress of power electronics technology, distributed generation technology is developing rapidly. Since micro grids cannot rely on traditional multi-time scale control strategies to ensure the high-quality frequency stability control and economic in the same time scale, this paper proposes an extreme dynamic programming algorithm. The proposed algorithm takes an adaptive dynamic programming algorithm as the framework, an extreme learning machine as a kernel of the evaluation module, a model module, an implementation module and a new prediction module. The resulting unified time scale intelligent control algorithm better realizes the combined functions of droop control + automatic generation control + economic dispatch in the traditional opermode. Finally, in order to verify the effectiveness of the proposed algorithm, a micro grid model of 8 nodes is simulated. The results confirm the feasibility and validity of the proposed extreme dynamic programming algorithm.

Modal analysis of a grid‐connected DFIG‐based WT considering multi‐timescale control interactions

In a grid-connected doubly fed induction generator (DFIG)-based wind turbine system, the multi-timescale control loops of DFIGs work together to keep the output active/reactive power tracking their references. In weak AC grid condition where the grid impedance cannot be neglected, interactions among those control loops are very strong. This study presents a modal analysis of the system containing a DFIG. The DFIG model for analysis considers multi-timescale control and its interactions. The effect of bandwidths of rotor speed control, phase-locked loop, DC-link voltage control and rotor current control on system stability is presented. Interactions among those control loops are explored. The analysis results indicate that the interactions among multi-timescale control are strong and will have an effect on the stability and the dynamics of the DFIG system under weak AC grid condition.