Data-driven Control Research Articles

Adaptive, Optimal, Virtual Synchronous Generator Control of Three-Phase Grid-Connected Inverters Under Different Grid Conditions—An Adaptive Dynamic Programming Approach

This article proposes an adaptive, optimal, data-driven control approach based on reinforcement learning and adaptive dynamic programming to the three-phase grid-connected inverter employed in virtual synchronous generators (VSGs). This article takes into account unknown system dynamics and different grid conditions, including balanced/unbalanced grids, voltage drop/sag, and weak grids. The proposed method is based on value iteration, which does not rely on an initial admissible control policy for learning. Considering the premise that the VSG control should stabilize the closed-loop dynamics, the VSG outputs are optimally regulated through the adaptive, optimal control strategy proposed in this article. Comparative simulations and experimental results validate the proposed method's effectiveness and reveal its practicality and implementation.

Data-driven control of spatiotemporal chaos with reduced-order neural ODE-based models and reinforcement learning

Deep reinforcement learning (RL) is a data-driven method capable of discovering complex control strategies for high-dimensional systems, making it promising for flow control applications. In particular, the present work is motivated by the goal of reducing energy dissipation in turbulent flows, and the example considered is the spatiotemporally chaotic dynamics of the Kuramoto–Sivashinsky equation (KSE). A major challenge associated with RL is that substantial training data must be generated by repeatedly interacting with the target system, making it costly when the system is computationally or experimentally expensive. We mitigate this challenge in a data-driven manner by combining dimensionality reduction via an autoencoder with a neural ODE framework to obtain a low-dimensional dynamical model from just a limited data set. We substitute this data-driven reduced-order model (ROM) in place of the true system during RL training to efficiently estimate the optimal policy, which can then be deployed on the true system. For the KSE actuated with localized forcing (‘jets’) at four locations, we demonstrate that we are able to learn a ROM that accurately captures the actuated dynamics as well as the underlying natural dynamics just from snapshots of the KSE experiencing random actuations. Using this ROM and a control objective of minimizing dissipation and power cost, we extract a control policy from it using deep RL. We show that the ROM-based control strategy translates well to the true KSE and highlight that the RL agent discovers and stabilizes an underlying forced equilibrium solution of the KSE system. We show that this forced equilibrium captured in the ROM and discovered through RL is related to an existing known equilibrium solution of the natural KSE.

안전하고 효과적인 자율주행을 위한 불확실성 순차 모델링

Deep reinforcement learning(RL) is an end-to-end data-driven control method that is widely used in the autonomous driving domain. However, conventional RL approaches have difficulties in applying it to autonomous driving tasks due to problems such as inefficiency, instability, and uncertainty. These issues play an important role in the autonomous driving domain. Although recent studies have attempted to solve these problems, they are computationally expensive and rely on special assumptions. In this paper, we propose a new algorithm MCDT that considers inefficiency, instability, and uncertainty by introducing a method called uncertainty sequence modeling to autonomous driving domain. The sequence modeling method, which views reinforcement learning as a decision making generation problem to obtain high rewards, avoids the disadvantages of exiting studies and guarantees efficiency, stability and also considers safety by integrating uncertainty estimation techniques. The proposed method was tested in the OpenAI Gym CarRacing environment, and the experimental results show that the MCDT algorithm provides efficient, stable and safe performance compared to the existing reinforcement learning method.

Data-Driven Variable Impedance Control of a Powered Knee-Ankle Prosthesis for Sit, Stand, and Walk with Minimal Tuning.

Although the average healthy adult transitions from sit to stand over 60 times per day, most research on powered prosthesis control has only focused on walking. In this paper, we present a data-driven controller that enables sitting, standing, and walking with minimal tuning. Our controller comprises two high level modes of sit/stand and walking, and we develop heuristic biomechanical rules to control transitions. We use a phase variable based on the user's thigh angle to parameterize both walking and sit/stand motions, and use variable impedance control during ground contact and position control during swing. We extend previous work on data-driven optimization of continuous impedance parameter functions to design the sit/stand control mode using able-bodied data. Experiments with a powered knee-ankle prosthesis used by a participant with above-knee amputation demonstrate promise in clinical outcomes, as well as trade-offs between our minimal-tuning approach and accommodation of user preferences. Specifically, our controller enabled the participant to complete the sit/stand task 20% faster and reduced average asymmetry by half compared to his everyday passive prosthesis. The controller also facilitated a timed up and go test involving sitting, standing, walking, and turning, with only a mild (10%) decrease in speed compared to the everyday prosthesis. Our sit/stand/walk controller enables multiple activities of daily life with minimal tuning and mode switching.

Deep Hierarchical Interval Type 2 Self-Organizing Fuzzy System for Data-Driven Robot Control

To solve the dimensional explosion problem, this paper proposes a new architecture for the fuzzy system, the deep hierarchical self-organizing interval type-2 fuzzy system (DHSOIT2FS). Each sub-fuzzy system is a self-organizing interval type-2 fuzzy system, constructed online, with rules constructed by a rule online update algorithm, consequent parameters updated by iterative least squares, and antecedent parameters are updated using a gradient descent algorithm. DHSOIT2FS uses a classic serial-layered structure to build the overall framework. The first layer uses the first two dimensions of data as input. Each subsequent layer uses the output of the previous layer with the next dimensional data as input until it is built. During the training process, each data point is trained with DHSOIT2FS before passing in the next data point to achieve online construction. The effectiveness of the approach in this paper is illustrated using two numerical simulation examples. The proposed method is also applied to a data-driven control example of a single-link robot and achieves good tracking results.

On the persistency of excitation

The result of J.C. Willems et al. “A note on persistency of excitation”, System & Control Letters, 2005 gives identifiability conditions for system identification as well as data-driven representations for data-driven control. The existing proofs however are proofs by contradiction and do not give insight into the assumptions of controllability and persistency of excitation of the input. Moreover, the existing proofs do not clarify how conservative the assumptions are. We provide an alternative constructive proof for the single-input case. It is shown that persistency of excitation of order more than the time horizon is needed in nongeneric cases, corresponding to special initial conditions. The special initial conditions are explicitly characterized in terms of the solution of a Sylvester equation. Another contribution of the paper is a representation of a scalar persistently exciting input of a finite order as an output of an autonomous linear time-invariant system.

Data-driven control for switched systems over a vulnerable and resource-constrained network

To alleviate the restriction of system model on control design, data-driven model-free adaptive control (MFAC) is an excellent alternative to model-based control methods. This paper studies event-triggered data-driven control for switched systems over a vulnerable and resource-constrained network. The system is transformed into an equivalent switched data model through dynamic linearization. Resource constraints and denial of service (DoS) attacks in the network are concerned, and a novel joint anti-attack method including resilient event-triggering mechanism and prediction scheme is presented. Furthermore, new event-triggered MFAC algorithms are proposed. In this scenario, by constructing a Lyapunov functional on tracking error, sufficient conditions to ensure its boundedness are derived. This is the first time in the literature to give a complete solution to data-driven control of switched systems. At last, the validity of new algorithms and theoretical results is confirmed by simulations.

A Regularized Real-Time Integrator for Data-Driven Control of Heating Channels

In many contexts of scientific computing and engineering science, phenomena are monitored over time and data are collected as time-series. Plenty of algorithms have been proposed in the field of time-series data mining, many of them based on deep learning techniques. High-fidelity simulations of complex scenarios are truly computationally expensive and a real-time monitoring and control could be efficiently achieved by the use of artificial intelligence. In this work we build accurate data-driven models of a two-phase transient flow in a heated channel, as usually encountered in heat exchangers. The proposed methods combine several artificial neural networks architectures, involving standard and transposed deep convolutions. In particular, a very accurate real-time integrator of the system has been developed.

Neural network-based event-triggered data-driven control of disturbed nonlinear systems with quantized input

This paper is devoted to design an event-triggered data-driven control for a class of disturbed nonlinear systems with quantized input. A uniform quantizer reconstructed with decreasing quantization intervals is employed to reduce the quantization error. A neural network-based estimation strategy is proposed to estimate both the pseudo partial derivative and disturbances. Consequently, an input triggering rule for single-input single-output systems is provided by incorporating the estimated disturbances, the quantization error bound and tracking errors. Resorting to the Lyapunov method, sufficient conditions for synthesized error systems to be uniformly ultimately bounded are presented. The validity of the proposed scheme is demonstrated via a simulation example.

A real-test and simulation combined platform for developing intelligent tracking control of multirotors

PurposeThis study aims to minimize the risk of costly failures of flight tests during the path tracking control design, and a noble approach has been proposed in this study to put the whole vehicle-in-the simulation loop. Working with the real system is essential for developing intelligent and data-driven controllers for multirotor drones which needs learning the drones' nonlinear complicated dynamics. The vehicle-in-the-loop (VIL) platform developed in this paper is a safe and effective solution to deal with this problem.Design/methodology/approachTo avoid risky flight test during controller design, the multirotor is hinged to a shaft that allows the multirotor's angular motion but restricts translational motion. The test-bed includes the real system attitude dynamics and the simulation of the position dynamics to model the complete flight based on real-time reactions of the vehicle. For the authors' case study, a hexacopter angular motion provides the real-time attitude data in translational motion simulation loop. To test the set-up, a proportional-integral-derivative (PID) and a brain emotional learning-based intelligent controller (BELBIC) is implemented for tracking of circle and 8-shape flight trajectories.FindingsThe results show that the platform helps the intelligent controller to learn the system dynamics without worrying about the failure in the early stages of the design and in the real-world flight test. Although the hexacopter translational dynamics is modeled in simulation, the authors still have highly accurate attitude dynamics matching the requirement of the control loop design. The comparison of the two controllers also shows that the performance of BELBIC is better than PID in this test.Originality/valueThe research background is reviewed in the introduction section. The other sections are originally developed in this paper.

Data-driven adaptive compensation control for a class of nonlinear discrete-time system with bounded disturbances

This paper considers the compensation control problem for a class of nonlinear discrete-time systems subject to bounded disturbances. With the help of the dynamic linearization technique (DLT), an equivalent data model to the unknown disturbed controlled plant is first established. Based on the data model, two data-driven controllers are designed through novel disturbance-related compact-form and partial-form DLT, which are equivalent to the unknown ideal compensation controller in theory. Adaptive gains designed for the proposed controllers are time-varying and are adaptively updated by directly utilizing the I/O data without involving any model information of the controlled plant, making both controllers purely data-driven adaptive disturbance compensation controllers. Further, in practice, unmeasurable disturbances are commonly encountered due to expensive measuring instruments, unreliable performance or large lags. Therefore, both proposed control laws provide solutions for measurable disturbance (MD) and unmeasurable disturbance (UD) in a unified framework, where the time-varying adaptive gains fuse more system dynamics when disturbance is completely unknown except for some boundedness. The stability of the proposed controllers are strictly guaranteed, and their effectiveness and applicability are verified by a numerical simulation and a distillation column.

Data-driven distributed voltage control for microgrids: A Koopman-based approach

This paper presents a distributed data-driven control to regulate the voltage in an alternate current microgrid (MG). Following the hierarchical control frame for MGs, a secondary control for voltage is designed with a data-driven strategy using the Koopman operator. The Koopman operator approach represents the nonlinear behavior of voltage as a linear problem in the space of observables or lifted space. The representation in the lifted space is used together with linear consensus to design a model predictive control (MPC). The complete algorithm is proved in an MG model that includes changes in load, transmission lines, and the communication graph. The data-driven model regulates the voltage using a distributed approach based only on local measurements, and includes reactive power constraints and control cost minimization.

Extremum-Seeking-Based Adaptive Model-Free Control and Its Application to Automated Vehicle Path Tracking

Traditional automated vehicle path-tracking algorithms require plant models to derive the respective control laws. However, the accurate vehicle model is difficult to obtain due to the complex tire-road interaction, time-varying parameters, and unknown disturbances. Consequently, data-driven controllers, which do not rely on a predefined plant model, have become increasingly popular. Notably, the model-free control (MFC), which expresses the derivatives of a controlled output as the sums of an amplified control input and an offset term, provides a straightforward solution to ground vehicle path tracking. Despite its ease of use, the control gain tuning of MFC remains largely a trial-and-error process, which could be both painstaking and poorly performing. Existing adaptive gain-tuning methods either rely on the command at the last step to iteratively update the control gain at the current step, or try to simultaneously identify the control gain and the offset term. However, they could yield chattering or unbounded control gains. To provide a new perspective for the MFC control gain tuning while improving MFC's control performance, we integrate MFC with the extremum-seeking control (ESC). ESC updates the control gain of MFC in real time to progressively enhance the control performance of MFC. Simulink–CarSim simulations and scaled car field tests demonstrate the effectiveness of the proposed extremum-seeking-based adaptive model-free controller.

The Wheelbot: A Jumping Reaction Wheel Unicycle

Combining off-the-shelf components with 3D- printing, the Wheelbot is a symmetric reaction wheel unicycle that can jump onto its wheels from any initial position. With non-holonomic and under-actuated dynamics, as well as two coupled unstable degrees of freedom, the Wheelbot provides a challenging platform for nonlinear and data-driven control research. This letter presents the Wheelbot's mechanical and electrical design, its estimation and control algorithms, as well as experiments <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^\star$</tex-math></inline-formula> demonstrating both self-erection and disturbance rejection while balancing.

Fractional data-driven model for stabilization of uncertain discrete-time nonlinear systems

This paper proposes a novel data-driven control for stabilization of a class of uncertain discrete-time nonlinear systems. The proposed method is based on the compact form dynamic linearization technique, which relates the first variation of the output signal with the fractional-order variation of the input one. Then, a discrete-time controller is proposed, based on the obtained fractional-order data-driven equivalent model. In order to compute the proposed controller and estimator, only input-output data information is considered. The uniform ultimately boundedness of the tracking errors are demonstrated by a formal analysis. Finally, comparison results based on simulations are presented to highlight the effectiveness of the proposed methodology.

Unfalsified Switching Adaptive Voltage Control for Islanded Microgrids

Microgrid's voltage regulation is of particular importance during both grid-connected and islanded modes of operation. Especially, during the islanded mode, when the support from the upstream grid is lost, stable voltage regulation is vital for the reliable operation of critical loads. This paper proposes a robust and data-driven control approach for secondary voltage control of AC microgrids in the presence of uncertainties. To this end, unfalsified adaptive control (UAC) is utilized to select the best stabilizing controller from a set of pre-designed controllers with the minimum knowledge required from the microgrid. Two microgrid test systems are simulated in MATLAB to verify the effectiveness of the proposed method under different scenarios like load change and communication link failure.

Data-driven control via Petersen’s lemma

We address the problem of designing a stabilizing closed-loop control law directly from input and state measurements collected in an experiment. In the presence of a process disturbance in data, we have that a set of dynamics could have generated the collected data and we need the designed controller to stabilize such set of data-consistent dynamics robustly. For this problem of data-driven control with noisy data, we advocate the use of a popular tool from robust control, Petersen’s lemma. In the cases of data generated by linear and polynomial systems, we conveniently express the uncertainty captured in the set of data-consistent dynamics through a matrix ellipsoid, and we show that a specific form of this matrix ellipsoid makes it possible to apply Petersen’s lemma to all of the mentioned cases. In this way, we obtain necessary and sufficient conditions for data-driven stabilization of linear systems through a linear matrix inequality. The matrix ellipsoid representation enables insights and interpretations of the designed control laws. In the same way, we also obtain sufficient conditions for data-driven stabilization of polynomial systems through alternate (convex) sum-of-squares programs. The findings are illustrated numerically.

Implementation of a robust data-driven control approach for an ommi-directional mobile manipulator based on koopman operator

The dynamic modeling and control of omni-directional mobile manipulators (OMM) are challenging since they are highly nonlinear, strongly coupled, and multi-input multi-output uncertainty systems. Koopman operator theory can provide an explicit linear dynamic model for OMM, only based on input-output data. However, the derived dynamic model usually has modeling errors and cannot capture external unknown disturbances. In this paper, a robust data-driven control scheme is designed and implemented for an OMM, based on Koopman operator and a disturbance observer. Firstly, a data-driven Koopman model of OMM is constructed, solely using the collected input-output data. Then a disturbance observer is designed to online estimate the inherent modeling error of the Koopman model as well as external disturbances. The controller is designed by combining linear MPC with feedback compensation of the estimated disturbances. Finally, both simulations and experimental tests are conducted to verify the effectiveness and robustness of the proposed control scheme.

Exploring the Design of Teaching Mental Health of College Students Based on Data-Driven Learning

Chinese contemporary college students, this generation has been pampered since childhood, growing up under the wings of their parents, most of them are flowers in the hothouse, now stepping into the university and carrying the double expectations of society and family. With the rapid development of modern technology and social culture, people in modern society are facing fierce competition, frequent stress, fast pace, and unprecedented psychological pressure, which has a significant impact on human health. Therefore, the construction of a university psychological wellness education model has become the focus of theoretical research. As a new type of mental health teaching and learning model, data-driven learning (DDL) not only provides learners with rich, diverse, and realistic mental health data, but also creates an ideal learning environment for learners due to its corpus-based teaching and learning characteristics. This paper explores the design of DDL-based mental health teaching and learning for students, combining theoretical research, and empirical analysis from the actual university, and constructing a comprehensive system of psychological wellness education in college while building a local system of DDL-based psychological wellness education. The experimental results show that the Q-learning algorithm and SA-Q algorithm have no environmental triggering mechanism, while the data-driven control algorithm has the step number of formula 24, 19, and 17, respectively, thus reaching the optimal path. Therefore, in the data-driven psychology classroom, students change from participating in activities according to the course teaching objectives and mental health teachers’ requirements to inquiry-based learning and interactive experiences that focuses on problem solving and task completion.

Online learning of data-driven controllers for unknown switched linear systems

Motivated by the goal of learning controllers for complex systems whose dynamics change over time, we consider the problem of designing control laws for systems that switch among a finite set of unknown discrete-time linear subsystems under unknown switching signals. To this end, we propose a method that uses data to directly design a control mechanism without any explicit identification step. Our approach is online, meaning that the data are collected over time while the system is evolving in closed-loop, and are directly used to iteratively update the controller. A major benefit of the proposed online implementation is therefore the ability of the controller to automatically adjust to changes in the operating mode of the system. We show that the proposed control mechanism guarantees stability of the closed-loop switched linear system provided that the switching is slow enough. Effectiveness of the proposed design technique is illustrated for two aerospace applications.