Model-free Control Research Articles

Model-Free Optimal Control of Inverter for Dynamic Voltage Support

Inverter-based resources (IBRs) are required to provide dynamic voltage support (DVS) during voltage dips to enhance the low-voltage ride-through capability. In this paper, a model-free control method is developed to achieve the optimal DVS (ODVS) without relying on the knowledge of grid parameters. Delving into the optimum trajectory of the ODVS problem, it is found that the current constraint and the maximum active power constraint of IBRs are both binding, or one of them is binding. This inspires us to search for the optimum in a closed-loop way by a perturb-and-observe (P&O)-based optimum seeking (OS) controller with either the power factor angle or the reactive current being the manipulated (perturbed) variable. The system is guaranteed to converge asymptotically to the optimum provided the stepsize sequence is diminishing and non-summable. The proposed model-free optimal control is finally implemented within a single-stage photovoltaic (PV) system, where dynamic simulations demonstrate the optimal and fast DVS performance. Moreover, the implementation strategy for other types of IBRs that are not self-protected by nature is also discussed.

Model-Free Predictive Control of Power Converters with Multifrequency Extended State Observers

The performance of model-free predictive control (MFPC), which is based on the ultra-local model, is highly dependent on the effectiveness of the extended-state observer (ESO). The standard linear ESO has a high-gain which enhances disturbance rejection attributes. Nevertheless, this ESO has two main limitations: i) it amplifies high-frequency measurement noises, and ii) it deteriorates disturbance rejection performance when the noise immunity is improved. Therefore, this article introduces a new technique to improve both the disturbance rejection and noise suppression of ESO without increasing the bandwidth: the multi-frequency-based ESO (MF-ESO). Two novel ESO structures that operate on the MF-ESO principle are analyzed: parallel and cascade ESOs. Extensive frequency domain analyses unravel the comparative capabilities of the MF-ESO structures for improved disturbance rejection and/or high-frequency noise suppression. A novel adaptive gain is proposed to improve CESO's dynamic performance. The superior features of the proposed MF-ESO are experimentally validated by the model-free predictive control of a grid-connected power converter.

Finite-Time Model-Free Adaptive Control for Discrete-Time Nonlinear Systems

This paper addresses the fast trajectory tracking problem for discrete-time nonlinear systems, and whereby a finite-time model-free adaptive control (FMFAC) approach is innovatively devised. Here, the nonlinear systems’ dynamics are completely unknown and the widespread disturbances are considered. The nonparametric dynamic linearization technique and the adaptive extended disturbance observer are firstly deployed to achieve real-time linearization of the discrete-time nonlinear system, and an efficient finite-time performance index function is thereafter devised, and thereby contributing to a novel FMFAC control scheme. Theoretical analysis indicates the finite-time model-free tracking control within a stable region can be guaranteed, and the boundedness of the control input signal can be ensured rigorously, even though under the effect of various disturbances. The key points of the proposed FMFAC approach are model-free and possessing the fast regulation ability, and the corresponding effectiveness and superiority are validated via simulation studies.

A Barrier-Certificated Reinforcement Learning Approach for Enhancing Power System Transient Stability

Increasing integration of renewable resources brings more flexibility and poses new challenges to modern power systems, leading to highly nonlinear and complex dynamics. This paper aims to provide a general solution framework to traditional control problems, such as frequency control and voltage control, which attempt to maintain the stability of either synchronous generators-governed or inverter-governed systems when subjected to a disturbance and simultaneously guarantee operational constraints, providing a complete complement to existing works on control design. Building on reinforcement learning (RL) and control barrier functions, the framework includes two subsystems, i.e., a model-free controller and a barrier-certification system, which discover RL-based control actions and sequentially filter them using a barrier certificate to satisfy operational constraints. Calculating a barrier function is generally challenging for a complex power system. This is addressed by representing the barrier function using neural networks (NNs) and data-based approaches. An adaptive method is introduced to certify the neural barrier function that perseveres barrier conditions, which is more compatible with online implementation. The proposed framework synthesizes a stabilizing controller that satisfies predefined safety regions. The effectiveness of the proposed framework is demonstrated via several comparative case studies.

An Online Model-Following Projection Mechanism Using Reinforcement Learning

In this paper, we propose a model-free adaptive learning solution for a model-following control problem. This approach employs policy iteration, to find an optimal adaptive control solution. It utilizes a moving finite-horizon of model-following error measurements. In addition, the control strategy is designed by using a projection mechanism that employs Lagrange dynamics. It allows for real-time tuning of derived actor-critic structures to find the optimal model-following strategy and sustain optimized adaptation performance. Finally, the efficacy of the proposed framework is emphasized through a comparison with sliding mode and high-order model-free adaptive control approaches.

Self-Healing Secure Blockchain Framework in Microgrids

Blockchain has recently been depicted as a secure protocol for information exchange in cyber-physical microgrids. However, it is still found vulnerable to consensus manipulation attacks. These stealth attacks are often difficult to detect as they use kernel-level access to mask their actions. In this paper, we firstly build a trusted and secured peer-to-peer network mechanism for physical DC microgrids’ validation of transactions over Distributed Ledger. Secondly, we leverage from a physics-informed approach for detecting malware-infected nodes and then recovering from stealth attacks using a self-healing recovery scheme augmented into the microgrid Blockchain network. This scheme allows compromised nodes to adapt to a reconstructed trustworthy signal in a multi-hop manner using corresponding measurements from the reliable nodes in the network. Additionally, recognizing the possible threat of denial-of-service attacks and random time delays (where information sharing via communication channels is blocked), we also integrate a model-free predictive controller with the proposed system that can locally reconstruct an expected version of the attacked/delayed signals. This supplements the capabilities of Blockchain, enabling it to detect and mitigate consensus manipulation attempts, and network latencies.

A Bayesian Deep Reinforcement Learning-Based Resilient Control for Multi-Energy Micro-Gird

Aiming at a cleaner future power system, many regimes in the world have proposed their ambitious decarboniz-ing plan, with increasing penetration of renewable energy sources (RES) playing an alternative role to conventional energy. As a re-sult, power system tends to have less backup capacity and operate near their designed limit, thus exacerbating system vulnerability against extreme events. Under this reality, resilient control for the multi-energy micro-grid is facing the following challenges, which are: 1) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">the effect from the stochastic uncertainties of</i> RES; 2) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">the need for a model-free and fast-reacting control scheme under extreme events; and</i> 3) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">efficient exploration and robust performance with limited extreme events data</i> . <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">To deal with the aforementioned challenges, this paper pro-poses a novel Bayesian Deep Reinforcement Learning-based resilient control approach for multi-energy micro-grid. In partic-ular, the proposed approach replaces the deterministic network in traditional Reinforcement Learning with a Bayesian probabilistic network in order to obtain an approximation of the value function distribution, which effectively solves the Q-value overestimation issue. Compared with the naive Deep Deterministic Policy Gra-dient (DDPG) method and optimization method, the effectiveness and importance of employing the Bayesian Reinforcement Learn-ing approach are investigated and illustrated across different operating scenarios. Case studies have shown that by using the Monte Carlo posterior mean of the Bayesian value function distribution instead of a deterministic estimation, the proposed Bayesian Deep Deterministic Policy Gradient (BDDPG) method achieves a near-optimum policy in a more stable process, which verifies the robustness and the practicability of the proposed approach.

Optimal model-free adaptive control based on reinforcement Q-Learning for solar thermal collector fields

Optimal model-free adaptive control based on reinforcement Q-Learning for solar thermal collector fields

Microvibration Control of Hybrid Platform for High-Tech Equipment in Buildings Using Brain Emotional Learning-Based Intelligent Controller

In the semiconductor industry, the vulnerability of high-tech facilities installed on platforms to ground excitations induced by nearby traffic is significantly pronounced, primarily due to their small-scale dimensions. Consequently, it is imperative to design a smart control technique by effectively utilizing model-free controllers. Recently, adaptive intelligent control algorithms have emerged as a viable alternative to conventional model-based control algorithms. To address this issue, this study meticulously designed a hybrid platform using the adaptive intelligent controller known as the brain emotional learning-based intelligent controller, along with a Sugano fuzzy inference system to effectively optimize the controller’s learning parameters. These learning and intelligent-based algorithms offer notable advantages, including the ability to handle nonlinearity, uncertainty, and training capabilities within the control systems. To assess the effectiveness of the proposed controller in mitigating vibrations induced by traffic on high-tech facilities, a three degree-of-freedom structure is employed along with the hybrid platform. Finally, the performance of the hybrid platform in terms of microvibration control levels is meticulously validated using the Bolt Beranek Newman vibration criteria. Simulation results unequivocally demonstrate that the proposed controller outperforms both an uncontrolled system and a traditional linear quadratic regulator controller in terms of reducing the traffic-induced response of the hybrid platform and second floor, respectively. Through the integration of learning and intelligent-based controllers, the velocity levels of both the hybrid platform and the second floor are reduced to approximately 49.02% and remain well within the acceptable standard criteria curves.

Model-free robust decentralised control of multi-input-multi-output nonlinear interconnected dynamic systems

This study presents a generalised framework for model-free robust decentralised control (MFRDC) of interconnected MIMO dynamic systems, with the goal of significantly reducing complexity in model-based design. A model-free robust sliding mode control, by Lyapunov differential inequality, is presented to achieve simultaneous nonlinear, dynamic, interaction/coupling inversion/cancellation (NDII) for such MIMO systems, which treats the plant/process as a total uncertainty from input to output. The U-control platform is presented, to integrate separately independently designed NDII and an invariant controller (IC) into a complete double loop control system. The associated robust stability and other properties are analysed to provide reference for applications. Two simulated tracking control tests are presented for functionally numerical demonstration, validation of the analytical results and illustration of the transparent procedure for general expansion/applications. These are: a coupled inverted pendulum and a two-input and two-output (TITO) non-affine nonlinear dynamic plant.

Reinforcement learning-based stable jump control method for asteroid-exploration quadruped robots

Unlike the spherical gravitational field of planets and other large solar system bodies, the gravitational field of asteroids is irregular and weak. It is challenging for a planetary rover to obtain sufficient traction forces in this environment. However, this gravitational environment is suitable for legged robots with jumping ability, but it also imposes higher demands on control methods. Therefore, this study aimed to address the problem of jump control method for asteroid-exploration quadruped robots. As the robot jumps off the surface of an asteroid, it would fly for a certain amount of time because of the low gravitational acceleration. The prolonged flight phase underscores the significance of the robot's take off and attitude control. A model-free stable jumping control method was devised in this study. This method can satisfy the control requirements for takeoff, attitude adjustment, and soft landing by using end-to-end multi-agent reinforcement learning (MARL). MARL is more advantageous than single-agent reinforcement learning in dealing with composite motion control problems under similar observation conditions. A simulated training environment was established, incorporating models of the gravitational field, task partitioning for jumping, and design of reward functions, including jump trajectory planning. The efficacy of the proposed jump control method for a quadruped robot was successfully demonstrated in the gravitational environment of an irregular rod-shaped asteroid, 216 Kleopatra.

Model-free control applied to a direct injection system: Experimental validation

This work presents the design and application of the Model-free controller with algebraic estimator and ultra-local model on the common-rail high-pressure in a Direct Injection Fuel System of an internal combustion engine. In this application, the PID controller is traditionally used, but it needs different settings as modifications in the operating point occur. These changes encompass motor rotation, injection time, or torque demand. On the other hand, this paper considers only a single set of control parameters that provides good performance considering a broad range of operating points. A commercial electronic control unit (FlexECU) offering the possibility of new software development was used to implement the control algorithm and manage all engine functions. The tuning process of the controller is presented and discussed. Tests were carried out, and the results were graphically demonstrated, evidencing the easy applicability, robustness, and ability of the considered Mode-free scheme to control nonlinear systems, even facing important dynamic variations.

Adaptive energy management strategy of plug-in hybrid electric bus

In this study, the online application of a new energy management strategy (EMS) is proposed for a power-split plug-in hybrid electric bus (PHEB) to improve its system's fuel economy and reduce power battery degradation. The new EMS employs Pontryagin's minimum principle (PMP) and the power battery state of charge (SOC) feedback method to adjust the covariates online. A model-free adaptive control (MFAC) EMS was designed and constructed according to the compact form dynamic linearization model to further obtain accurate SOC feedback; the time–domain cumulative as well as covariate upper and lower limit constraints were used to obtain a real-time MFAC-Ⅲ strategy for online applications. The average reduction in battery capacity loss on all ten routes was 8.19 %, resulting from the correlation between the MFAC-III EMS and MFAC EMS; evaluations conducted on both strategies indicated that the MFAC-III outperforms the MFAC strategy in hardware-in-the-loop.

Learning to Control a Three-Dimensional Ferrofluidic Robot.

In recent years, ferrofluids have found increased popularity as a material for medical applications, such as ocular surgery, gastrointestinal surgery, and cancer treatment, among others. Ferrofluidic robots are multifunctional and scalable, exhibit fluid properties, and can be controlled remotely; thus, they are particularly advantageous for such medical tasks. Previously, ferrofluidic robot control has been achieved via the manipulation of handheld permanent magnets or in current-controlled electromagnetic fields resulting in two-dimensional position and shape control and three-dimensional (3D) coupled position-shape or position-only control. Control of ferrofluidic liquid droplet robots poses a unique challenge where model-based control has been shown to be computationally limiting. Thus, in this study, a model-free control method is chosen, and it is shown that the task of learning optimal control parameters for ferrofluidic robot control can be performed using machine learning. Particularly, we explore the use of Bayesian optimization to find optimal controller parameters for 3D pose control of a ferrofluid droplet: its centroid position, stretch direction, and stretch radius. We demonstrate that the position, stretch direction, and stretch radius of a ferrofluid droplet can be independently controlled in 3D with high accuracy and precision, using a simple control approach. Finally, we use ferrofluidic robots to perform pick-and-place, a lab-on-a-chip pH test, and electrical switching, in 3D settings. The purpose of this research is to expand the potential of ferrofluidic robots by introducing full pose control in 3D and to showcase the potential of this technology in the areas of microassembly, lab-on-a-chip, and electronics. The approach presented in this research can be used as a stepping-off point to incorporate ferrofluidic robots toward future research in these areas.

Model-Free HVAC Control in Buildings: A Review

The efficient control of HVAC devices in building structures is mandatory for achieving energy savings and comfort. To balance these objectives efficiently, it is essential to incorporate adequate advanced control strategies to adapt to varying environmental conditions and occupant preferences. Model-free control approaches for building HVAC systems have gained significant interest due to their flexibility and ability to adapt to complex, dynamic systems without relying on explicit mathematical models. The current review presents the recent advancements in HVAC control, with an emphasis on reinforcement learning, artificial neural networks, fuzzy logic control, and their hybrid integration with other model-free algorithms. The main focus of this study is a literature review of the most notable research from 2015 to 2023, highlighting the most highly cited applications and their contributions to the field. After analyzing the concept of each work according to its control strategy, a detailed evaluation across different thematic areas is conducted. To this end, the prevalence of methodologies, utilization of different HVAC equipment, and diverse testbed features, such as building zoning and utilization, are further discussed considering the entire body of work to identify different patterns and trends in the field of model-free HVAC control. Last but not least, based on a detailed evaluation of the research in the field, the current work provides future directions for model-free HVAC control considering different aspects and thematic areas.

Transfer learning for occupancy-based HVAC control: A data-driven approach using unsupervised learning of occupancy profiles and deep reinforcement learning

Model-free heating, ventilation, and air conditioning (HVAC) control systems have demonstrated promising potential for adjusting indoor setpoint temperature based on dynamic occupancy patterns in smart buildings. Although these control systems offer the advantage of not needing building or occupancy models, the involved trial-and-error learning process can cause considerable thermal discomfort for occupants, particularly during the initial learning period. Given the critical importance of thermal comfort, this limitation is a major barrier to the practical implementation of such systems. To address this challenge, the present study proposes a framework to enhance the learning process of the model-free HVAC controllers. Specifically, a transfer learning (TL) technique is adopted based on a similarity analysis of occupancy patterns using an unsupervised learning of occupancy profiles. This control framework leverages a k-means clustering algorithm with dynamic time warping to match the most similar households in terms of occupancy patterns within 26 residential units. The results demonstrate that the proposed method significantly improves the performance of the HVAC control system. It enhances the jumpstart performance and total rewards by nearly 25% and 5%, respectively, compared to a conventional model-free controller. Furthermore, it reduces the deviation period and mean temperature deviation by approximately 4% and 68%, respectively. Overall, this framework presents a promising approach to enhancing the performance and practicality of model-free HVAC control systems by reducing the thermal discomfort during the learning process.

Improved cascaded model-free predictive speed control for PMSM speed ripple minimization based on ultra-local model

This paper presents an improved cascaded model-free predictive speed and current control with the periodic and aperiodic disturbances suppression to achieve a smooth speed. The cascaded structure has an external speed loop and an internal current loop, both implemented with model-free predictive control to enhance the robustness of the controller. The current loop is designed based on the finite control set model-free predictive current control (FCS-MFPCC) strategy with an ultra-local model to regulate the stator currents, and the speed loop uses the proposed continuous control set model-free predictive speed control (CCS-MFPSC) to make full use of the excellent dynamic performance of the current-loop controller. To suppress the periodic disturbance that exists in the PMSM system, an improved parallel quasi-resonant controller (QRC) with an error limitation is embedded into the CCS-MFPSC, which can generate the compensated current. Based on the stability condition, the stability of the proposed MFPSC-QRC strategy is directly analyzed in the z-domain. Finally, the effectiveness and feasibility of the proposed cascaded model-free predictive speed and current strategy are validated on a PMSM test platform.

Effective Energy Management Strategy with Model-Free DC-Bus Voltage Control for Fuel Cell/Battery/Supercapacitor Hybrid Electric Vehicle System

This article presents a new design method of energy management strategy with model-free DC-Bus voltage control for the fuel-cell/battery/supercapacitor hybrid electric vehicle (FCHEV) system to enhance the power performance, fuel consumption, and fuel cell lifetime by considering regulation of DC-bus voltage. First, an efficient frequency-separating based-energy management strategy (EMS) is designed using Harr wavelet transform (HWT), adaptive low-pass filter, and interval type–2 fuzzy controller (IT2FC) to determine the appropriate power distribution for different power sources. Second, the ultra-local model (ULM) is introduced to re-formulate the FCHEV system by the knowledge of the input and output signals. Then, a novel adaptive model-free integral terminal sliding mode control (AMFITSMC) based on nonlinear disturbance observer (NDO) is proposed to force the actual values of the DC-link bus voltage and the power source’s currents track their obtained reference trajectories, wherein the NDO is used to approximate the unknown dynamics of the ULM. Moreover, the Lyapunov theorem is used to verify the stability of AMFITSMC via a closed-loop system. Finally, the FCHEV system with the presented method is modeled on a Matlab/Simulink environment, and different driving schedules like WLTP, UDDS, and HWFET driving cycles are utilized for investigation. The corresponding simulation results show that the proposed technique provides better results than the other methods, such as operational mode strategy and fuzzy logic control, in terms of the reduction of fuel consumption and fuel cell power fluctuations.

Model free position-force control of environmental constrained reconfigurable manipulators based on adaptive dynamic programming

Model free position-force control of environmental constrained reconfigurable manipulators based on adaptive dynamic programming

Robust adaptive path following control of autonomous underwater vehicle with uncertainties and communication bandwidth limitation

This article investigates the path-following control problem for autonomous underwater vehicles (AUVs) subject to external disturbances and unknown model parameters. An event-triggered model-free adaptive control (ETMFAC) method is designed with a practical experimental platform. Firstly, a line-of-sight (LOS) guidance law is employed to obtain the desired heading angle for the AUV. Secondly, an improved model-free adaptive control method is used, in which an adaptive law is introduced to compensate for the effects caused by external disturbances. Subsequently, an event-triggering mechanism that can reduce the communication and calculation burden of the control system is proposed. As for the state mutation caused by variable disturbances, an adaptive triggering threshold is designed based on the state vector of the AUV to maintain satisfactory following performance. The stability of the control system is proved by theoretical analysis. Finally, both numerical simulations and practical experiments are conducted to evaluate the efficacy and prove the superiority of the proposed method.