Data-driven Control Research Articles

Data-driven control of micro-climate in buildings: An event-triggered reinforcement learning approach

AbstractSmart buildings have great potential for shaping an energy-efficient, sustainable, and more economic future for our planet as buildings account for approximately 40% of the global energy consumption. Future of the smart buildings lies in using sensory data for adaptive decision making and control that is currently gloomed by the key challenge of learning a good control policy in a short period of time in an online and continuing fashion. To tackle this challenge, an event-triggered – as opposed to classic time-triggered – paradigm, is proposed in which learning and control decisions are made when events occur and enough information is collected. Events are characterized by certain design conditions and they occur when the conditions are met, for instance, when a certain state threshold is reached. By systematically adjusting the time of learning and control decisions, the proposed framework can potentially reduce the variance in learning, and consequently, improve the control process. We formulate the micro-climate control problem based on semi-Markov decision processes that allow for variable-time state transitions and decision making. Using extended policy gradient theorems and temporal difference methods in a reinforcement learning set-up, we propose two learning algorithms for event-triggered control of micro-climate in buildings. We show the efficacy of our proposed approach via designing a smart learning thermostat that simultaneously optimizes energy consumption and occupants’ comfort in a test building.

Using a dynamically selective support vector data description model to discover novelties in the control system of electric arc furnace

As increasing data-driven control strategies are applied in electric arc furnace systems, the problem of novelty detection has drawn more attentions than before. The presence of outliers should be the main obstacle in practical applications for these advanced control techniques. To this end, this paper proposes a dynamically selective support vector data description model to discover novelties in electric arc furnace. In this model, support vector data description plays the role of base detector. Artificial outliers are generated with two objectives, one is to assist the dynamic selection, and the other is to optimize two parameters of support vector data description. Then clustering technique is used to determine the validation set for each test point. Finally, a probabilistic method is used to compute the competence of base detectors. In contrast to other novelty ensembles that have parallel structures, our ensemble model has a dynamic selection mechanism that could facilitate the mining of the potential of base detectors. Three synthetic and three real-world datasets are used to validate the effectiveness of the proposed detection model. Experimental results have approved our method by comparing it with several competitors.

Data-Driven Tests for Controllability

The fundamental lemma due to Willems et al. “A note on persistency of excitation,” Syst. Control Lett., vol. 54, no. 4, pp. 325-329, 2005 plays an important role in system identification and data-driven control. One of the assumptions for the fundamental lemma is that the underlying linear time-invariant system is controllable. In this letter, the fundamental lemma is extended to address system identification for uncontrollable systems. Then, a data-driven algebraic test is derived to check whether the underlying system is controllable or not. An algorithm based on the singular value decomposition of a Hankel matrix constructed from the data is provided to implement the developed test. The algorithm has cubic computational cost. Examples are given to illustrate the theoretical results.

A Semi-Algebraic Optimization Approach to Data-Driven Control of Continuous-Time Nonlinear Systems

This letter considers the problem of designing state feedback data-driven controllers for nonlinear continuous-time systems. Specifically, we consider a scenario where the unknown dynamics can be parametrized in terms of known basis functions and the available measurements are corrupted by unknown-but-bounded noise. The goal is to use this noisy experimental data to directly design a rational state-feedback control law guaranteed to stabilize all plants compatible with the available information. The main result of this letter shows that, by using Rantzer's Dual Lyapunov approach, combined with elements from convex analysis, the problem can be recast as an optimization over positive polynomials, which can be relaxed to a semi-definite program through the use of Sum-of-Squares and semi-algebraic optimization arguments. Three academic examples are considered to illustrate the effectiveness of the proposed method.

A Data-Driven Storage Control Framework for Dynamic Pricing

Dynamic pricing is both an opportunity and a challenge to the end-users. It is an opportunity as it better reflects the real-time market conditions and hence enables an active demand side. However, demand's active participation does not necessarily lead to benefits. The challenge conventionally comes from the limited flexible resources and limited intelligent devices on the demand side. The decreasing cost of the storage system and the widely deployed smart meters inspire us to design a data-driven storage control framework for dynamic prices. Our work first establishes a stylized model by assuming the knowledge on the structure of dynamic price distributions and designs the optimal storage control policy. Based on Gaussian Mixture Model, we propose a practical data-driven control framework, which helps relax the assumptions in the stylized model. Numerical studies illustrate the remarkable performance of the proposed data-driven framework.

Dynamic variation in dew-point temperature of attached air layer of radiant ceiling cooling panels

The present article describes the integration of a data-driven predictive demand response control for residential buildings with heat pump and on-site energy generation. The data driven control approach schedules the heating system of the building. In each day, the next 24 hours heating demand of buildings, including space heating and domestic hot water consumption, are predicted by means of a hybrid wavelet transformation and a dynamic neural network. Linear programming is implemented to define a cost-optimal schedule for the heat pump operation. Moreover, the study discusses the impact of heat demand prediction error on performance of demand response control. In addition, the option of energy trading with the electrical grid is considered in order to evaluate the possibility of increasing the profit for private householders through on-site energy generation. The results highlight that the application of the proposed predictive control could reduce the heating energy cost up to 12% in the cold Finnish climate. Furthermore, on-site energy generation declines the total energy cost and consumption about 43% and 24% respectively. The application of a data-driven control for the demand prediction brings efficiency to demand response control.

Enhanced model-free adaptive iterative learning control with load disturbance and data dropout

In this paper, an enhanced model-free adaptive iterative learning control (EMFAILC) method is proposed, which is applied for a class of nonlinear discrete-time systems with load disturbance and random data dropout. This method is a data-driven control strategy and only the I/O data are required for the controller design. Data are lost at every time instance and iteration instance independently, which allows successive data dropout both in time and iterative axes. By compensating the missing data, the proposed EMFAILC algorithm can track the desired time-varying trajectory. The convergence and effectiveness of the proposed approach are verified by both the rigorous mathematical analysis and the simulation results.

Perimeter Control of Urban Traffic Networks Based on Model-Free Adaptive Control

In urban traffic networks, abundant traffic data is generated everyday. Therefore, data-driven control approaches for urban traffic control without utilizing mathematical models might be a promising research direction. In this paper, a hierarchical perimeter control strategy for urban traffic networks based on model-free adaptive control (MFAC) scheme is proposed. At the outer level, the signal settings in the periphery of the congested region are optimized to meter the number of vehicles entering the congested region; while at the inner level, traffic flow distribution within the congested region is adjusted to improve the traffic efficiency. MFAC scheme is applied in both outer level and inner level to derive the detailed signal settings in the congested region and in the periphery of the congested region. Joint simulations using VISSIM and MATLAB verify the feasibility and effectiveness of the proposed method.

Data-driven Sliding Mode Control for Pulses of Fluorescence in STED Microscopy Based on Förster Resonance Energy Transfer Pairs

ABSTRACTPairs of conjugate donor-acceptor fluorescent probes have proven themselves useful in stimulated emission depletion (STED) microscopy in recent years. For instance, it has been shown that the lifetime of said probes directly correlates to the resolution of the microscope. However, once the lifetimes of the probes have been optimized, it is desirable to control their fluorescence in order to improve the resolution further. Here, we propose combining model-free control with sliding mode control to track nanosecond pulses of red-shifted acceptor fluorescence in order to inhibit visible light emitted from the image plane, shrink the point spread function, and subsequently improve the resolution of the microscope. This is achieved by automatic adjustment of the STED laser beam pump power. This controller is numerically simulated against a generic model created from Förster resonance energy transfer (FRET) theory. However, since it is data-driven, it can be easily applied to various physical systems with drastically different dynamics. This work provides a reliable theoretic control solution to modern super resolution microscopy for biological imaging.

Ultra-local model predictive control: A model-free approach and its application on automated vehicle trajectory tracking

Model predictive control (MPC) has been extensively utilized in the automotive applications, such as autonomous vehicle path planning and control, hybrid-vehicle energy management, and advanced driver-assistance system design. As a typical model-based control law, MPC relies on a system model to predict the state evolution of the manipulated plant within the prediction horizon. However, a representative yet concise mathematical description of the controlled plant may not always be available in practice. Therefore, model-free strategies, e.g., identification for control and direct data-driven control, have been incorporated into the predictive control framework. Nonetheless, existing model-free predictive controllers usually require reliable datasets and employ complex nonconvex optimizations to identify the underlying system model. Furthermore, their control performances are fundamentally limited by the quality of the training data. Inspired by the model-free control, this paper proposes the ultra-local model predictive control (ULMPC), which is a novel and straightforward model-free predictive control technique with no need for the computationally-extensive model learning process. The proposed ULMPC is implemented for automated vehicle trajectory following. Carsim-Simulink joint simulations and indoor experimental field tests with a scaled car demonstrate its effectiveness.

Grasp Posture Control of Wearable Extra Robotic Fingers with Flex Sensors Based on Neural Network

This study proposes a data-driven control method of extra robotic fingers to assist a user in bimanual object manipulation that requires two hands. The robotic system comprises two main parts, i.e., robotic thumb (RT) and robotic fingers (RF). The RT is attached next to the user’s thumb, while the RF is located next to the user’s little finger. The grasp postures of the RT and RF are driven by bending angle inputs of flex sensors, attached to the thumb and other fingers of the user. A modified glove sensor is developed by attaching three flex sensors to the thumb, index, and middle fingers of a wearer. Various hand gestures are then mapped using a neural network. The input data of the robotic system are the bending angles of thumb and index, read by flex sensors, and the outputs are commanded servo angles for the RF and RT. The third flex sensor is attached to the middle finger to hold the extra robotic finger’s posture. Two force-sensitive resistors (FSRs) are attached to the RF and RT for the haptic feedback when the robot is worn to take and grasp a fragile object, such as an egg. The trained neural network is embedded into the wearable extra robotic fingers to control the robotic motion and assist the human fingers in bimanual object manipulation tasks. The developed extra fingers are tested for their capacity to assist the human fingers and perform 10 different bimanual tasks, such as holding a large object, lifting and operate an eight-inch tablet, and lifting a bottle, and opening a bottle cap at the same time.

A Review of Research on Intersection Control Based on Connected Vehicles and Data-Driven Intelligent Approaches

Benefiting from the application of vehicle communication networks and new technologies, such as connected vehicles, video monitoring, automated vehicles and vehicle–road collaboration, traffic network data can be observed in real-time. Applied in the field of traffic control, these technologies can provide high-quality input data and make a more comprehensive evaluation of the effectiveness of traffic control. However, most of the control theories and strategies adopted by adaptive control systems cannot effectively use these real-time, high-precision data. In order to adapt to the development of the times, intersection control theory needs to be further developed. This paper reviews the intersection control strategies from many perspectives, including intelligent data-driven control, conventional timing control, induction control and model-based traffic control. There are three main directions for intersection control based on the connected vehicle environment: (1) data-driven reinforcement learning control; (2) adaptive performance optimization control; (3) research on traffic control based on the environment of connected vehicles (CV); and (4) multiple intersection control based on the CV environment. The review gives a clear view of the data-driven intelligent control theory and its application for intelligent transportation systems.

Design of a Data-Driven Multi PID Controllers using Ensemble Learning and VRFT

Data-driven control has been proposed for directly calculating control parameters using experimental data. Specifically, the Virtual Reference Feedback Tuning (VRFT) has been proposed for linear time-invariant systems. In the field of machine learning, the ensemble learning was proposed to improve the accuracy of prediction by using multiple learners. In this study, a design scheme of data-driven controllers using the ensemble learning and VRFT is newly proposed for linear time-varying systems. The ensemble learning can divide the linear time-varying system into some sections that can be regarded locally as linear time-invariant systems.

IoT-Based DC/DC Deep Learning Power Converter Control: Real-Time Implementation

Recently, a modularized smart grid (SG) architecture, entitled the Internet of Things (IoT) grid, is developed that accommodates the IoT technology into the dc-dc converters to build a programmable grid with a single voltage bus. This modern architecture can be established with low computing hardware that facilitates the control and management of the IoT-based grids. Due to the uncertainties originated from the integration of the IoT technology and power electronic converters, the deterministic methodologies are unable to precisely model the SG anymore. In response to these challenges, this article addresses a novel adaptive data-driven method based on the active disturbance rejection controller (ADRC) for the voltage regulation of an IoT-based dc-dc buck converter feeding constant power loads. In particular, a deep deterministic policy gradient (DDPG) with the actor-critic architecture is adopted for the online adjusting of the ADRC controller. The established DDPG takes into account the ADRC controller coefficients into the design objective and offers the ADRC controller with the online coefficient setting ability through the neural network learning. The IoT-based system is tested on a real-time testbed with the constrained application protocol protocol and IEEE 802.11 (Wi-Fi) network to assess the applicability of the suggested controller in the presence of network degradations. The impact of both packet loss and interfering traffic on the reduction performance of the DDPG adaptive ADRC controller is investigated, simultaneously. The supremacy of the suggested adaptive data-driven controllers is verified by a comprehensive comparative analysis with the state-of-the-art methodologies.

OPTICS

Lawns, also known as turf, cover an estimated 128,000 km 2 in North America alone, with landscape requirements representing 30% of freshwater consumed in the residential domain. With this consumption comes a large amount of environmental, economic, and social incentive to make turf irrigation systems as efficient as possible. Recent work introduced the concept of distributed control in irrigation systems, but existing control strategies either do not take advantage of the distributed control, or do not revise the strategy over time in response to collected data. In this work, we introduce OPTICS, a data-driven control strategy that self-improves over time, adapts to the local specific conditions and weather changes, and requires virtually no human input in both setup and maintenance providing a plug-and-play system that requires minimal pre-deployment efforts. In addition to substantial improvements in ease-of-use, we find across 4 weeks of large-scale irrigation system deployment that OPTICS improves system efficiency by 12.0% in comparison to industry best and 3.3% in comparison to academic state of the art. Despite using less water, OPTICS also was found to improve quality of service by a factor of 4.0× compared to industry best and 2.5× compared to academic state of the art.

Willems’ Fundamental Lemma for State-Space Systems and Its Extension to Multiple Datasets

Willems et al.'s fundamental lemma asserts that all trajectories of a linear system can be obtained from a single given one, assuming that a persistency of excitation and a controllability condition hold. This result has profound implications for system identification and data-driven control, and has seen a revival over the last few years. The purpose of this letter is to extend Willems' lemma to the situation where multiple (possibly short) system trajectories are given instead of a single long one. To this end, we introduce a notion of collective persistency of excitation. We will show that all trajectories of a linear system can be obtained from a given finite number of trajectories, as long as these are collectively persistently exciting. We will demonstrate that this result enables the identification of linear systems from data sets with missing samples. Additionally, we show that the result is of practical significance in data-driven control of unstable systems.

Polynomial regressors based data-driven control for autonomous underwater vehicles

This paper proposes a data-driven control approach on the basis of polynomial regressors for autonomous underwater vehicles (AUVs). In contrast to conventional control approach, data-driven control does not require modeling for the systems. It only utilizes the analysis of massive stored dataset to predict the future control input, which can achieve the future output. Generally, the massive stored dataset can be analyzed by short-length vectors linearly. In this paper, a novel point is the improvement of existing data-driven control by polynomial regressors, which improve the control performance of AUVs. By numerical simulations, we illustrate the effectiveness of our proposed approach.

On the Sample Complexity of Data-Driven Inference of the L2-Gain

Lately, data-driven control has become a widespread area of research. A few recent big-data based approaches for data-driven control of nonlinear systems try to use classical input-output techniques to design controllers for systems for which only a finite number of (input-output) samples are known. These methods focus on using the given data to compute bounds on the $\mathcal{L}_2$-gain or on the shortage of passivity from finite input-output data, allowing for the application of the small gain theorem or the feedback theorem for passive systems. One question regarding these methods asks about their sample complexity, namely how many input-output samples are needed to get an approximation of the operator norm or of the shortage of passivity. We show that the number of samples needed to estimate the operator norm of a system is roughly the same as the number of samples required to approximate the system in the operator norm.

A deep reinforcement learning method for managing wind farm uncertainties through energy storage system control and external reserve purchasing

In deregulated environment, the wind power producers (WPPs) will face the challenge of how to increase their revenues under uncertainties of wind generation and electricity price. This paper proposes a method based on deep reinforcement learning (DRL) to address this issue. A data-driven controller that directly maps the input observations, i.e., the forecasted wind generation and electricity price, to the control actions of the wind farm, i.e., the charge/discharge schedule of the relevant energy storage system (ESS) and the reserve purchase schedule, is trained according to the method. By the well-trained controller, the influence of the uncertainties of wind power and electricity price on the revenue can be automatically involved and an expected optimal decision can be obtained. Furthermore, a targeted DRL algorithm, i.e., the Rainbow algorithm, is implemented to improve the effectiveness of the controller. Especially, the algorithm can overcome the limitation of the conventional reinforcement learning algorithms that the input states must be discrete, and thus the validity of the control strategy can be significantly improved. Simulation results illustrate that the proposed method can effectively cope with the uncertainties and bring high revenues to the WPPs.

A data-driven predictive controller combined with the vector autoregressive with exogenous input model and the propagator estimation method for vehicle lateral stabilization

A general predictive controller based on the subspace model identification method is proposed for vehicle stabilization. Traditional predictive controllers are always developed based on the principle model of vehicles, which inevitably suffers from parameter uncertainty and poor adaptability. In contrast to that, the proposed subspace-based general predictive controller is realized by a data-driven process and presents good adaptability in vehicle stability control. Inspired by subspace-based predictor construction, the keys of the predictive controller are as follows: (1) system model identification according to the model structure of the control object by input and output data; (2) output prediction of the system by the identified model; and (3) optimal control law designed by combining the linear–quadratic–Gaussian index with the predictive output. The main problem in the controller development lies in the recursive estimation of relevant matrices, which is limited by the subspace model identification theory. The implementation of the vector autoregressive with exogenous input model and the propagator method in subspace identification algorithm effectively solves the problem of estimation accuracy and calculation efficiency. Combined with a linear–quadratic–Gaussian index function, the predictive law for vehicle stability control is derived in detail. Finally, based on the vehicle model validated by standard road test, the effectiveness and robustness of the predictive controller are proved through the numerical simulations of various maneuvers under different road adhesive conditions.