Suboptimal Controller Research Articles

Approximate solution of a zero-sum linear-quadratic differential game by the auxiliary parameter method: analytical/numerical study

ABSTRACT A zero-sum finite horizon linear-quadratic differential game is considered. First, the terminal-value problem for the game-theoretic Riccati matrix differential equation, associated with the considered game by the solvability conditions, is analysed. This problem may not have the solution in the entire time-interval of the game's duration. Using the method of auxiliary parameter, a sufficient condition (in the terms of the game's data) for the existence of the solution to the aforementioned terminal-value problem in the entire time-interval of the game's duration is derived. An approximate analytical solution to this problem also is obtained as a partial sum of some infinite series of functions arising in the method of auxiliary parameter. Based on this approximate solution, suboptimal state-feedback controls of the players are formally designed and justified. It is shown that the pair of these controls constitutes an approximate-saddle point in the considered game. The theoretical results of the article are illustrated by their application to approximate solution of the problem of pursuit-evasion engagement between two flying vehicles.

Dynamic event-triggered finite-horizon robust suboptimal control of multi-player systems with input disturbances

This paper presents an adaptive dynamic event-triggered robust finite-horizon suboptimal control scheme for multi-player nonzero-sum game of nonlinear systems with input disturbances based on dynamic programming (ADP) and integral sliding mode (ISM) control techniques. Firstly, dynamic event-triggered adaptive ISM controllers are designed to handle input disturbances that can force the system state to remain on the sliding manifold and relax the known upper bounds condition of input disturbances. Corresponding event-triggered condition is employed for ISM controller of each player to reduce the update frequency. Then, utilizing the obtained ISM dynamics, we employ ADP-based single critic neural networks to approximate the time-varying solution of the Hamilton–Jacobi equations. Thus, the event-triggered finite-horizon suboptimal controllers can be obtained, and corresponding state-based dynamic event-triggered condition is designed to improve the resource utilization. The stability property of the closed-loop system is demonstrated through the Lyapunov technique. Finally, the effectiveness of the proposed control scheme is confirmed through two simulation examples.

Some new results about the stability radius of infinite-dimensional systems perturbed stochastically and deterministically

This paper presents two significant contributions: First, we examine linear infinite-dimensional systems perturbed in deterministic and stochastic ways. The stability radius method is used to solve the robust stability. Bounds on the perturbations are obtained, guaranteeing that the troubled system is stable. The outcomes are determined via a Lyapunov equation. Secondly, we study the robust stabilisation problem. We investigate the maximisation of the stability radius when the input operator is unbounded in the general case. Also, we establish the conditions under which suboptimal controllers exist. An infinite-dimensional Riccati equation describes the supreme achievable stability radius.

Cardioprotective Effects of SGLT2 Inhibitors in Diabetic Patients with Cardiovascular Disease

Type 2 diabetes is a chronic and multifactorial disease associated with a twofold increase in the incidence of numerous cardiovascular and renal diseases, which are a significant health limitation for patients with diabetes. Throughout the years, cardiovascular diseases have marked a significant health burden in these patients. Many studies have shown that Sodium-glucose cotransporter 2 (SGLT2) inhibitors reduce cardiovascular mortality in patients with diabetes mellitus, by inhibiting cardiomyocyte apoptosis. Patients with diabetes are at a higher mortality risk after their first event of myocardial infarction than those without diabetes. The keystone in the therapy of diabetic patients is significantly linked to the prevention of cardiovascular events. The high prevalence of cardiovascular and renal disease in patients with diabetes, as well as suboptimal glycemic controls and the significance of cardiovascular and renal risk reduction in type 2 diabetes, suggest that SGLT2 inhibitors have a significant clinical advantage, enhancing not just a better glycemic control, but improving cardiovascular and renal outcomes.

Observer-Based Suboptimal Controller Design for Permanent Magnet Synchronous Motors: State-Dependent Riccati Equation Controller and Impulsive Observer Approaches

Permanent Magnet Synchronous Motors (PMSMs) with high energy efficiency, reliable performance, and a relatively simple structure are widely utilised in various applications. In this paper, a suboptimal controller is proposed for PMSMs without sensors based on the state-dependent Riccati equation (SDRE) technique combined with customised impulsive observers (IOs). Here, the SDRE technique facilitates a pseudo-linearised display of the motor with state-dependent coefficients (SDCs) while preserving all its nonlinear features. Considering the risk of non-available/non-measurable states in the motor due to sensor and instrumentation costs, the SDRE is combined with IOs to estimate the PMSM speed and position states. Customised IOs are proven to be capable of obtaining quality, continuous estimates of the motor states despite the discrete format of the output signals. The simulation results in this work illustrate an accurate state estimation and control mechanism for the speed of the PMSM in the presence of load torque disturbances and reference speed changes. It is clearly shown that the SDRE-IO design is superior compared to the most popular existing regulators in the literature for sensorless speed control.

Data‐driven adaptive optimal control for discrete‐time periodic systems

AbstractIn this paper, a problem of data‐driven optimal control is studied for discrete‐time periodic systems with unknown system matrices and input matrices. For this problem, a value iteration‐based adaptive dynamic programming algorithm is proposed to obtain the suboptimal controller. The core of the algorithm proposed in this paper is to obtain an approximation of the unique positive definite solution of the algebraic Riccati equation and the optimal feedback gain matrix by using the collected real‐time data of the system states and control inputs. Without an initial stabilizing feedback gain, the proposed algorithm could be activated by an arbitrary bounded control input. Finally, the effectiveness of the proposed approach is demonstrated by two examples.

Stochastic Optimal Linear Control for Generalized Cost Functions With Time-Invariant Stochastic Parameters.

This study presents the design of feedback controllers for generalized cost functions to deal with stochastic optimal control problems. Target linear systems contain time-invariant stochastic parameters that describe system uncertainty. The cost functions involve nonlinear mappings and polynomial forms of the system states and inputs to express various performance metrics. Unfortunately, these properties cause difficulties in solving the problems. Conventional methods, such as the principle of optimality, are not employed to solve such problems owing to the time-invariant parameters. As opposed to the well-known quadratic functions, handling the generalized cost functions is a complicated task. This study overcomes these challenges by deriving an explicit relation between the cost function and the linear feedback gain of a controller. The derived relation enables the feedback gain to be optimized via a gradient method. A theoretical analysis ensures the convergence of the proposed gradient method. A suboptimal feedback controller is obtained to solve the problem, even for the generalized cost. Furthermore, the controller guarantees robust stability of the feedback system even with the stochastic parameters. It is demonstrated that the proposed cost function can express an expectation of a quadratic cost, risk-sensitive cost, polynomial cost, and input-to-state gain. A numerical simulation shows the effectiveness of the proposed method.

Design of Generalized H∞-Suboptimal Controllers Based on Experimental and A Priori Data

This paper considers a linear continuous- or discrete-time dynamic object in the absence of its mathematical model. As is demonstrated below, a control law that suboptimally damps initial and (or) exogenous disturbances of such objects can be implemented based on experimental and a priori data. The approach involves the methods of robust control design and duality theory as well as the technique of linear matrix inequalities.

Design of Generalized H∞-Suboptimal Controllers Based on Experimental and A Priori Data

Показано, что при отсутствии математической модели линейного непрерывного или дискретного динамического объекта закон управления, обеспечивающий субоптимальное гашение начального и/или внешнего возмущений, может быть реализован по экспериментальным и априорным данным. В основе подхода лежат методы построения робастного управления, теория двойственности и аппарат линейных матричных неравенств.

Design of Generalized H∞-Suboptimal Controllers Based on Experimental and a Priori Data

Design of Generalized H∞-Suboptimal Controllers Based on Experimental and a Priori Data

Efficient LQG control for wireless networked control systems without packet acknowledgement

This paper studies the linear quadratic Gaussian (LQG) control problem for wireless networked control systems (WNCSs) without packet acknowledgement. In WNCSs, packets may be randomly lost in both control and observation channels, and non-acknowledgement (non-ACK) mechanisms (e.g. UDP-like protocols) are generally adopted to achieve real-time control and reduce energy consumption. For general non-ACK systems, it is well known that the optimal controller is not yet available, and the performance of existing suboptimal controllers is poor because they either adopt a fixed control law or rely on a poorly performing filter. By using a moment-preserving Gaussian approximation approach, an efficient filter is proposed to approximately estimate system states. Based on the approximate state estimates of the proposed filter, an approximate optimal controller with a time-varying control gain for WNCSs is further designed by using a dynamic programming approach. The proposed controller has excellent performance and becomes the optimal controller under special system parameters. Numerical simulations show the correctness and effectiveness of the obtained results.

Deployment of real-time building automation system-integrated inverse-model-based fault detection and diagnostics algorithms

The complex operation of HVAC systems in large commercial buildings warrants regular implementation of advanced analytical approaches to operations and maintenance, and subsequent corrective measures to improve and maintain optimal energy performance. Despite the established capabilities of data-driven fault detection and diagnostics (FDD) to identify suboptimal controls policies and mechanical faults resulting in poor energy performance, few attempts have been made to deploy scalable solutions around these approaches. Furthermore, real-time BAS-integrated FDD methods are predominantly rule-based, offering limited insights to faults with gradual negative impacts to energy performance. This paper demonstrates the application of various established data-driven, inverse-model-based FDD methodologies in a BAS-integrated environment. Traditionally implemented sparingly, the novelty of recursive and automatic execution of advanced FDD methodologies, facilitated through a direct data pipeline to an existing BAS, capitalizes on the BAS’s real-time monitoring capabilities to enable continuously refreshed inverse model generation that can capture the gradual degradation of building performance, and provide up-to-date actionable visualizations and key performance indicators (KPI) to building operators. Since deployment, the application has successfully identified a scheduling fault on two separate occasions in a case study building in Ottawa, Canada, and the visualizations were presented to the building operators who resolved the issues.

Calibrating subjective data biases and model predictive uncertainties in machine learning-based thermal perception predictions

Heating, Ventilation, and Air Conditioning (HVAC) systems in large-scale buildings often struggle to ensure satisfactory thermal comfort for diverse occupants while minimizing energy waste. Achieving this goal requires developing reliable prediction models that capture the changing and varied occupant thermal perceptions in different spaces. Despite their widespread use, many machine learning (ML) based prediction models suffer from subjective data biases and model predictive uncertainties, causing inaccurate estimation for occupant needs and leading to suboptimal building controls. The authors propose a data-model integration method that identifies and calibrates the inherent uncertainties of existing ML models in both data and model dimensions, ensuring reliable thermal perception predictions. This method introduces the Multidimensional Association Rule Mining (M-ARM) algorithm to identify biased human responses by exploring interrelationships among four perception metrics: thermal sensation, comfort, acceptability, and preference. Our method reveals significant performance enhancements in seven ML models, enhancing the F1-score by up to 5.53%. By leveraging reliability diagrams and Expected Calibration Error (ECE) scores, we also expose the models’ vulnerability to miscalibration and the need for calibrated predictions. We further evaluate six calibration techniques (e.g., Platt Scaling and Isotonic Calibration) on these models and uncover their potential to enhance prediction reliability performance, highlighting a reduction of up to 80.66% in ECE scores. The authors also investigated the impacts of dataset sizes, classifiers, and calibration methods on the proposed method. Our research offers insight into creating robust data-driven strategies for thermal perception predictions, ultimately contributing to optimized occupant comfort and energy efficiency in buildings.

Polymorphic Virtual Synchronous Generator: An Advanced Controller for Smart Inverters

Virtual synchronous generators (VSGs) are one of the most relevant solutions to integrate renewable energy in weak grids and microgrids. They indeed provide inverters characteristics of rotating machines (inertia for instance) that are useful for stabilizing the system, notably in the context of the high variability of the production. Thanks to the virtual characteristics of the VSG, the virtual parameters of the emulated synchronous machine can be optimally adapted online as a function of the electric environment of the inverter. We call that inverter’s control a polymorphic VSG. The online adaptation of the critical control parameters of the VSG helps reduce the risk of deterioration of the inverter’s constituents that might be induced by harsh events (frequent in weak grids) but, more importantly, improves the robustness of the system. In this paper, four implementations of a polymorphic VSG controller are compared on a simple microgrid study case to a complete VSG model. For the test, polymorphic VSGs have to minimize frequency and voltage oscillations while withstanding short circuits, which is typically a requirement for units in this context. One of the controls is based on recurrent optimization over a prediction time horizon, and two sub-optimal ones target practical implementation in industrial inverters with limited computational power. Results show a clear reduction in incidents in the microgrid thanks to the controllers. The error reduction with the complete polymorphic VSG is up to 100% for the voltage, 32% for the currents, and 79% for the duty ratio. Those values are decreased by 30 to 50% with the sub-optimal controllers but for a reduction in the computational burden of more than 97%. Recommendations are proposed for the development of an auto-adaptive polymorphic VSG from a high technology-readiness-level perspective, i.e., targeting a compromise between error reduction and computational burden.

Discrete Uncertainty Quantification For Offline Reinforcement Learning

Abstract In many Reinforcement Learning (RL) tasks, the classical online interaction of the learning agent with the environment is impractical, either because such interaction is expensive or dangerous. In these cases, previous gathered data can be used, arising what is typically called Offline RL. However, this type of learning faces a large number of challenges, mostly derived from the fact that exploration/exploitation trade-off is overshadowed. In addition, the historical data is usually biased by the way it was obtained, typically, a sub-optimal controller, producing a distributional shift from historical data and the one required to learn the optimal policy. In this paper, we present a novel approach to deal with the uncertainty risen by the absence or sparse presence of some state-action pairs in the learning data. Our approach is based on shaping the reward perceived from the environment to ensure the task is solved. We present the approach and show that combining it with classic online RL methods make them perform as good as state of the art Offline RL algorithms such as CQL and BCQ. Finally, we show that using our method on top of established offline learning algorithms can improve them.

Robust stability region analysis of time‐delayed load frequency control systems with EVs aggregator using Kharitonov theorem

AbstractThis study focuses on analyzing the robust stability regions and robustness margin of a time‐delayed load frequency control (LFC) system with Electric Vehicles (EVs) using the Kharitonov Theorem. Communication time delays in LFC systems can jeopardize stability and reliability, leading to suboptimal controller performance. Integrating EVs into LFC systems enhances dynamical stability but introduces additional complexity. Therefore, it is crucial to employ robust analysis and controller design techniques to ensure system stability. In this study, we utilize the Kharitonov Theorem to determine the robust stability regions and stability boundaries in the parameter plane of the Proportional‐Integral (PI) controller. By considering communication time delays and parametric uncertainties in the LFC system with EVs (LFC‐EVs), the robust PI controller gains using these methods are efficiently computed. To evaluate the performance of the theoretically computed robust controller parameters, time‐domain simulations are conducted.

Design of Suboptimal Robust Controllers Based on a Priori and Experimental Data

Design of Suboptimal Robust Controllers Based on a Priori and Experimental Data

Suboptimal control law for a multi fractional high order linear quadratic regulator system in the presence of disturbance

A new method for finding a suboptimal control law for a multi fractional spring–mass system as a fractional high order linear quadratic regulator system in the presence of known disturbance is introduced. The method is presented in a general form for solving practical multi fractional linear quadratic regulator systems with disturbances. After defining a closed-loop controller, formulations for finding the suboptimal controller are derived. In the proposed method, there is no need to solve the fractional vector differential equation. Simulation studies demonstrate the applicability of the method for practical purposes. Some new types of a spring–mass–damper system are considered.

A Data-Driven Solution for Energy Management Strategy of Hybrid Electric Vehicles Based on Uncertainty-Aware Model-Based Offline Reinforcement Learning

Energy management strategy (EMS) is the key technology to improving the fuel efficiency of hybrid electric vehicles (HEV). In recent years, the development of artificial intelligence has enabled tremendous advances by utilizing reinforcement learning (RL) for training and deploying deep neural network-based EMS. However, in contrast to the fields of deep learning such as computer vision and natural language processing which mainly rely on large-scale offline datasets, most RL-based policies must be trained online by trial-and-error with the initial performance being almost arbitrary. Such a paradigm is considered inefficient and unsafe for industrial automation and can only be used to tackle the EMS problems in the simulation world. Considering large historical interactive datasets are readily available in EMS domain, if a RL algorithm can be used to extract a policy purely offline from the prior collected dataset and improve upon data logging policy, the current issues including sample inefficiency, unsafe exploration, and simulation-to-real gap that prevent the widespread use of RL methods could be mitigated to a great extent. To this end, this paper presents a feasible algorithmic framework for model-based offline RL. Unlike vanilla RL approaches without any consideration against distributional shift, a data-driven dynamic model is built before the policy training using RL. After that, two techniques namely conservative MDP and state regularization are augmented which is proved to be effective against model over exploitation. A hardware-in-the-loop (HIL) test is used to evaluate the real-time performance and effectiveness of different strategies on a real controller, and it is found that the proposed naïve data-driven model-based offline RL algorithm can already solve the distributional shift problem inherent in vanilla RL to a large extent. Furthermore, by incorporating the guidance of uncertainty-awareness, a near optimal policy can be obtained by using only the dataset from a sub-optimal controller.

Reinforcement Learning-Based Near Optimization for Continuous-Time Markov Jump Singularly Perturbed Systems

The design of a suboptimal controller for continuous-time Markov jump singularly perturbed systems with partially unknown dynamics is studied in this paper. With fast and slow decomposition technique, the original Markov jump singularly perturbed systems are decomposed into fast and slow subsystems as a new attempt. On this basis, an offline parallel Kleinman algorithm and an online parallel integral reinforcement learning algorithm are presented to cope with the different subsystems, respectively. Meanwhile, the controllers obtained by the above two algorithms are used to design the suboptimal controllers for original systems. Furthermore, the suboptimality of the proposed controllers is also discussed. Finally, an example of the electric circuit model is shown to illustrate the applicability of the proposed method.