Benchmark Control Problem Research Articles

Intrinsic Motivation in Dynamical Control Systems

Biological systems often choose actions without an explicit reward signal, a phenomenon known as intrinsic motivation. The computational principles underlying this behavior remain poorly understood. In this study, we investigate an information-theoretic approach to intrinsic motivation, based on maximizing an agent's empowerment (the mutual information between its past actions and future states). We show that this approach generalizes previous attempts to formalize intrinsic motivation, and we provide a computationally efficient algorithm for computing the necessary quantities. We test our approach on several benchmark control problems, and we explain its success in guiding intrinsically motivated behaviors by relating our information-theoretic control function to fundamental properties of the dynamical system representing the combined agent-environment system. This opens the door for designing practical artificial, intrinsically motivated controllers and for linking animal behaviors to their dynamical properties. Published by the American Physical Society 2024

Adaptive sliding-mode delay compensation for real-time hybrid simulations with multiple actuators

Real-time hybrid simulation (RTHS) is a widely applied test method in structural engineering, which is developed from pseudo-dynamic test. Much of the past work has been centered on one-dimensional RTHS using a single hydraulic actuator. When the complexity of the problem demands to increase the number of degrees of freedom to be enforced on the boundary conditions, more than one hydraulic actuator must be used. Multiple-actuator or multi-axial RTHS (maRTHS) requires that more than one hydraulic actuator exerts the required motion on experimental substructures demanding the implementation of multiple-input multiple-output (MIMO) control strategies. A new maRTHS benchmark control problem has been developed, focusing on a frame subjected to seismic load at the base, substantially transforming and intensifying the complexity of the problem. The time delay generated by the dynamic characteristics of the loading system and the transmission process as well as the high coupling between the hydraulic actuators and the nonlinear kinematics escalates the complexity of the actuator control tracking. A sliding mode adaptive delay compensation method suitable for maRTHS is proposed, which utilizes a MIMO sliding mode method to reduce the coupling effects of actuators and the adaptive compensation method to compensate the residual delay. The effectiveness of the method is verified by numerical simulating different working conditions in the Benchmark Problem Platform.

Reinforcement learning based MPC with neural dynamical models

This paper presents an end-to-end learning approach to developing a Nonlinear Model Predictive Control (NMPC) policy, which does not require an explicit first-principles model and assumes that the system dynamics are either unknown or partially known. The paper proposes the use of available measurements to identify a nominal Recurrent Neural Network (RNN) model to capture the nonlinear dynamics, which includes constraints on the state variables and inputs. To address the issue of suboptimal control policies resulting from simply fitting the model to the data, this paper uses Reinforcement learning (RL) to tune the NMPC scheme and generate an optimal policy for the real system. The approach’s novelty lies in the use of RL to overcome the limitations of the nominal RNN model and generate a more accurate control policy. The paper discusses the implementation aspects of initial state estimation for RNN models and integration of neural models in MPC. The presented method is demonstrated on a classic benchmark control problem: cascaded two tank system (CTS).

Conditional adaptive time series compensation and control design for multi-axial real-time hybrid simulation

The structural performance of critical infrastructure during extreme events requires testing to understand the complex dynamics. Shake table testing of buildings to evaluate structural integrity is expensive and requires special facilities that can allow for the construction of large-scale test specimens. An attractive alternative is a cyber-physical testing technique known as Real-Time Hybrid Simulation (RTHS), where a large-scale structure is decomposed into physical and numerical substructures. A transfer system creates the interface between physical and numerical substructures. The challenge occurs when using multiple actuators connected with a coupler (i.e., transfer system) to create translation and rotation at the interface. Tracking control strategies aim to reduce time delay errors to create the desired displacements that account for the complex dynamics. This paper proposes two adaptive control methodologies for multi-axial real-time hybrid simulations that improve capabilities for a higher degree of coupling, boundary, complexity, and noise reduction. One control method integrates the feedback proportional derivative integrator (PID) control with a conditional adaptive time series (CATS) compensation and inverse decoupler. The second proposed control method is based on a coupled Model Predictive Control (MPC) with the CATS compensation. The performance of the proposed methods is evaluated using the virtual multi-axial benchmark control problem consisting of a steel frame as the experimental substructure. The transfer system consists of a coupler that connects two hydraulic actuators generating the translation and rotation acting at the joint. Through sensitivity analysis, parameters were tuned for the decoupler components, CATS compensation, and the control design for PID, LQG, and MPC. Comparative results among different control methods are evaluated based on performance criteria, including critical factors such as reduction in the time delay of bothactuators. The research findings in this paper improve the tracking control systems for the multi-axial RTHS of building structures subjected to earthquake loading. It provides insight into the robustness of the proposed tracking control methods in addressing uncertainty and improves the understanding of multiple output controllers that could be used in future cyber-physical testing of civil infrastructure subjected to natural hazards.

Learning Lyapunov terminal costs from data for complexity reduction in nonlinear model predictive control

AbstractA classic way to design a nonlinear model predictive control (NMPC) scheme with guaranteed stability is to incorporate a terminal cost and a terminal constraint into the problem formulation. While a long prediction horizon is often desirable to obtain a large domain of attraction and good closed‐loop performance, the related computational burden can hinder its real‐time deployment. In this article, we propose an NMPC scheme with prediction horizon and no terminal constraint to drastically decrease the numerical complexity without significantly impacting closed‐loop stability and performance. This is attained by constructing a suitable terminal cost from data that estimates the cost‐to‐go of a given NMPC scheme with long prediction horizon. We demonstrate the advantages of the proposed control scheme in two benchmark control problems.

Adaptive Neuroevolution With Genetic Operator Control and Two-Way Complexity Variation

Topology and weight evolving artificial neural network (TWEANN) algorithms optimize the structure and weights of artificial neural networks (ANNs) simultaneously. The resulting networks are typically used as policy models for solving control and reinforcement learning (RL) type problems. This paper presents a neuroevolution algorithm that aims to address the typical stagnation and sluggish convergence issues present in other neuroevolution algorithms. These issues are often caused by inadequacies in population diversity preservation, exploration/exploitation balance, and search flexibility. This new algorithm, called the Adaptive Genomic Evolution of Neural-Network Topologies (AGENT), builds on the neuroevolution of augmenting topologies (NEAT) concept. Novel mechanisms for adapting the selection and mutation operations are proposed to favorably control population diversity and exploration/exploitation balance. The former is founded on a fundamentally new way of quantifying diversity by taking a graph-theoretic perspective of the population of genomes and inter-genomic differences. Further advancements to the NEAT paradigm occur through the incorporation of variable neuronal properties and new mutation operations that uniquely allow both the growth and pruning of ANN topologies during evolution. Numerical experiments with benchmark control problems adopted from the OpenAI Gym illustrate the competitive performance of AGENT against standard RL methods and adaptive HyperNEAT, and superiority over the original NEAT algorithm. Further parametric analysis provides key insights into the impact of the new features in AGENT. This is followed by evaluation on an unmanned aerial vehicle collision avoidance problem where maneuver planning models are learnt by AGENT with 33% reward improvement over 15 generations.

Experimental benchmark control problem for multi-axial real-time hybrid simulation

Advancing RTHS methods to readily handle multi-dimensional problems has great potential for enabling more advanced testing and synergistically using existing laboratory facilities that have the capacity for such experimentation. However, the high internal coupling between hydraulics actuators and the nonlinear kinematics escalates the complexity of actuator control and boundary condition tracking. To enable researchers in the RTHS community to develop and compare advanced control algorithms, this paper proposes a benchmark control problem for a multi-axial real-time hybrid simulation (maRTHS) and presents its definition and implementation on a steel frame excited by seismic loads at the base. The benchmark problem enables the development and validation of control techniques for tracking both translation and rotation degrees of freedom of a plant that consists of a steel frame, two hydraulic actuators, and a steel coupler with high stiffness that couples the axial displacements of the hydraulic actuators resulting in the required motion of the frame node. In this investigation, the different components of this benchmark were developed, tested, and a set of maRTHS were conducted to demonstrate its feasibility in order to provide a realistic virtual platform. To offer flexibility in the control design process, experimental data for identification purposes, finite element models for the reference structure, numerical, and physical substructure, and plant models with model uncertainties are provided. Also, a sample example of an RTHS design based on a linear quadratic Gaussian controller is included as part of a computational code package, which facilitates the exploration of the tradeoff between robustness and performance of tracking control designs. The goals of this benchmark are to: extend existing control or develop new control techniques; provide a computational tool for investigation of the challenging aspects of maRTHS; encourage a transition to multiple actuator RTHS scenarios; and make available a challenging problem for new researchers to investigate maRTHS approaches. We believe that this benchmark problem will encourage the advancing of the next-generation of controllers for more realistic RTHS methods.

A New Approach to Detecting and Preventing Populations Stagnation Through Dynamic Changes in Multi-Population-Based Algorithms

Abstract In this paper, a new mechanism for detecting population stagnation based on the analysis of the local improvement of the evaluation function and the infinite impulse response filter is proposed. The purpose of this mechanism is to improve the population stagnation detection capability for various optimization scenarios, and thus to improve multi-population-based algorithms (MPBAs) performance. In addition, various other approaches have been proposed to eliminate stagnation, including approaches aimed at both improving performance and reducing the complexity of the algorithms. The developed methods were tested, among the others, for various migration topologies and various MPBAs, including the MNIA algorithm, which allows the use of many different base algorithms and thus eliminates the need to select the population-based algorithm for a given simulation problem. The simulations were performed for typical benchmark functions and control problems. The obtained results confirm the validity of the developed method.

Application of recurrent neural networks for modeling and control of a quadruple‐tank system

AbstractThe quadruple tank (QT) system consists of four interacting tanks and can switch between the minimum and non‐minimum phase behavior with changes in the positions of pump valves and is considered a benchmark control problem. In the present study, long‐short term memory (LSTM), a type of recurrent neural networks (RNN) is designed for the benchmark QT system based on the model‐based control framework. Random input–output sequences are generated from the white box model of the QT system to train an LSTM network model. The LSTM network is tuned by adjusting its hyperparameters such as the number of hidden layers, hidden units, and epochs to minimize the prediction error on the test data. The trained model is cross validated both during and after training to avoid overfitting. Once a reasonably reliable model is obtained, another LSTM network is trained for use as a controller. The network architecture is constantly modified till the controller is able to track the test setpoints with minimum error. This procedure is repeated with a gated recurrent unit (GRU) network and the servo and regulatory response of both the network models and controller are evaluated in terms of standard performance measure namely root mean square error (RMSE), integral square error (ISE), and control effort (CE). It is observed that the controller designed based on RNN performs better than a conventional centralized controller.

Gauss–Newton meets PANOC: A fast and globally convergent algorithm for nonlinear optimal control

PANOC is an algorithm for nonconvex optimization that has recently gained popularity in real-time control applications due to its fast, global convergence. The present work proposes a variant of PANOC that makes use of Gauss–Newton directions to accelerate the method. Furthermore, we show that when applied to optimal control problems, the computation of this Gauss–Newton step can be cast as a linear quadratic regulator (LQR) problem, allowing for an efficient solution through the Riccati recursion. Finally, we demonstrate that the proposed algorithm is more than twice as fast as the traditional L–BFGS variant of PANOC when applied to an optimal control benchmark problem, and that the performance scales favorably with increasing horizon length.

Metaheuristic Optimization Algorithm for Optimal Design of Type-2 Fuzzy Controller

The utilization of Le`vy flight to create new candidate solutions is one of the most powerful elements of CS. Candidate solutions are modified using this method by making a lot of minor modifications and a few big jumps. As a result, CS will be able to significantly increase the link between exploration and exploitation while also improving its search capabilities. The cuckoo search optimization (CSO) algorithm is applied to interval type-2 fuzzy logic controller (IT2FLC) in this research to determine the optimal parameters of membership functions (MFs) of interval type-2 fuzzy logic systems (IT2FLSs). The study takes into account two forms of MFs: triangular and trapezoidal. When perturbations are applied during the execution of each control issue, the CSO algorithm's performance and efficiency improve significantly. The proposed approach is tested using two benchmark control problems: water tank controller and inverted pendulum controller.

Sampling Rate Optimization and Execution Time Analysis for Two-Degrees-of-Freedom Control Systems

The current journal paper proposes an end-to-end analysis for the numerical implementation of a two-degrees-of-freedom (2DOF) control structure, starting from the sampling rate selection mechanism via a quasi-optimal manner, along with the estimation of the worst-case execution time (WCET) for the specified controller. For the sampling rate selection, the classical Shannon–Nyquist sampling theorem is replaced by an optimization problem that encompasses the trade-off between the fidelity of the controllers’ representation, along with the fidelity of the resulting closed-loop systems, and the implementation difficulty of the controllers. Additionally, the WCET analysis can be seen as a verification step before automatic code generation, a computational model being provided. The proposed computational model encompasses infinite-impulse response (IIR) and finite-impulse response (FIR) filter models for the controller implementation, along with additional relevant phenomena being discussed, such as saturation, signal scaling and anti-windup techniques. All proposed results will be illustrated on a DC motor benchmark control problem.

Neural H₂ Control Using Continuous-Time Reinforcement Learning.

In this article, we discuss continuous-time H2 control for the unknown nonlinear system. We use differential neural networks to model the system, then apply the H2 tracking control based on the neural model. Since the neural H2 control is very sensitive to the neural modeling error, we use reinforcement learning to improve the control performance. The stabilities of the neural modeling and the H2 tracking control are proven. The convergence of the approach is also given. The proposed method is validated with two benchmark control problems.

Predictions Based on Evolutionary Algorithms Using Predefined Control Profiles

The general motivation of our work is to meet the main time constraint when implementing a control loop: the Controller’s execution time is less than the sampling period. This paper proposes a practical method to diminish the computational complexity of the controllers using predictions based on the Evolutionary Algorithm (EA). It is the case of Model Predictive Control or, more generally, Receding Horizon Control structures. The main drawback of the metaheuristic algorithms (including EAs) working in control structures is their great complexity. Usually, the control variables take values between minimum and maximum technological limits. This work’s main idea is to consider the control variables’ domain inside a predefined control profile’s neighbourhood. The Controller takes into account a smaller domain of the control variables without tracking the predefined control profile or a reference trajectory. The convergence of the EA under consideration is not affected; hence, the same best predictions are found. The predefined control profile is already known or can be determined by solving the optimal control problem without time constraints in open-loop and offline. This work also presents a simulation study applying the proposed technique that involves two benchmark control problems. The results prove that the computational complexity decreases significantly.

Interval type-2 generalized fuzzy hyperbolic modelling and control of nonlinear systems

Generalized Fuzzy Hyperbolic Models (GFHMs) offer a simpler structure and less computational complexity than the typical fuzzy systems. Type-2 fuzzy systems, in contrast, have better handling of uncertainty but at the cost of higher computational complexity. Here, we propose a synergistic hybrid framework of interval type-2 fuzzy systems and GFHMs for a better uncertainty handling and simpler computational structure in the modelling and control of nonlinear systems. For this purpose, we first extend the GFHM to a computational model with various width and subsequently propose interval type-2 generalized fuzzy hyperbolic systems (IT2-GFHS) as a computational framework for nonlinear systems modelling. We then employ this IT2-GFHS in a general sliding-based robust nonlinear controller. Theoretical Lyapunov analysis reveals the overall asymptotic stability of the resulting closed-loop system. The numerical simulations for system modelling and identification on two nonlinear benchmark problems also reveal higher accuracy, lower computation time, and fewer adjustable parameters for the proposed IT2-GFHS models. Furthermore, applications to two nonlinear benchmark control problems show similar performance in terms of robustness to noise and disturbances compared with type-2 fuzzy systems, with the IT2-GFHS-based nonlinear controller having considerably fewer computations and floating-point operations. Finally, the proposed approach is experimentally implemented to control a 3-Prismatic-Series-Prismatic (3-PSP) parallel robot. Experimental results also confirm the improved tracking performance of the proposed method compared with interval type-2 and type-1 fuzzy systems, while also requiring fewer adjustable parameters.

Analysis of a Cart-Inverted Pendulum System with Harmonic Disturbances Based on its Criterion Matrix

The control of an inverted pendulum is a classical benchmark control problem. Its dynamics resemble that of many real-world systems of interest like pendulous, missile launchers, segways, and many more. The control of this system is challenging as it is a highly unstable, highly non-linear, non-minimum phase system, and underactuated. Furthermore, the physical constraints on the track position also pose complexity in its control design. A great deal of nonlinearity is present inherently and as well as affected by the surrounding external disturbances. The paper presents an approach for analysis of a cart-inverted pendulum system with harmonic disturbances. The approach is based on the index of the criterion matrix of the system named a degeneration factor. The degeneration factor is constructed with the singular values of the criterion matrix of the system and allows us to find frequency range, where the system operates as a whole. A linear-quadratic regulator is used to control the cart-inverted pendulum system. The results are supported with an example.

Modified ant colony optimization based PID controller design for coupled tank system

Most of the optimization algorithm suffers from slow convergence and poor solution accuracy due to nonlinearity characteristics of the problem. This paper proposes a meta-heuristic method, namely the modified Ant Colony Optimization (m-ACO) technique to address these issues. The conventional ant colony optimization algorithm is improved by using multiple random initializations to increase the probability of getting a better start population and hence increase the possibility to achieve near-global optimum. The efficacy of the proposed algorithm is tested on classical benchmark functions and on the PID controllers. The performance of controllers depends on the selection of tuning parameters that are optimized to obtain an optimum solution. The m-ACO technique is used to optimize parameters of PID controller for Coupled Tank System. It is one of the benchmark control problems due to its nonlinear characteristics. The obtained results of the proposed algorithm are compared with the Zeigler Nichols tuned PID controller, and PSO tuned PID controller in terms of transient performances including rise-time, peak overshoot and performance parameters such as Integral Square Error (ISE) and Integral Absolute Error (IAE). The proposed method is also tested for showing effectiveness in set-point tracking and disturbance rejection.

Differential Evolution with Shadowed and General Type-2 Fuzzy Systems for Dynamic Parameter Adaptation in Optimal Design of Fuzzy Controllers

This work is mainly focused on improving the differential evolution algorithm with the utilization of shadowed and general type 2 fuzzy systems to dynamically adapt one of the parameters of the evolutionary method. Previously, we have worked with both kinds of fuzzy systems in different types of benchmark problems and it has been found that the use of fuzzy logic in combination with the differential evolution algorithm gives good results. In some of the studies, it is clearly shown that, when compared to other algorithms, our methodology turns out to be statistically better. In this case, the mutation parameter is dynamically moved during the evolution process by using shadowed and general type-2 fuzzy systems. The main contribution of this work is the ability to determine, through experimentation in a benchmark control problem, which of the two kinds of the used fuzzy systems has better results when combined with the differential evolution algorithm. This is because there are no similar works to our proposal in which shadowed and general type 2 fuzzy systems are used and compared. Moreover, to validate the performance of both fuzzy systems, a noise level is used in the controller, which simulates the disturbances that may exist in the real world and is thus able to validate statistically if there are significant differences between shadowed and general type 2 fuzzy systems.

Nonlinear control using human behavior learning

In this paper, we discuss a human behavior learning approach for nonlinear systems control . We use several cognitive models and human skills to model and accelerate the learning process. A neural reinforcement learning algorithm is applied as main cognitive model. A persistent exciting signal and experience replay methods are proposed to improve learning accuracy and overcome the sensitivity problem of the human actions. The stability and convergence of the neural network based reinforcement learning is discussed. Simulations results verify the approach with two benchmark control problems.

An Efficient Chicken Search Optimization Algorithm for the Optimal Design of Fuzzy Controllers

A proposed architecture to design the optimal parameters of Membership Functions (MFs) of Type-1 Fuzzy Logic Systems (T1FLSs) using the Chicken Search Optimization (CSO) is applied to three Fuzzy Logic Controllers (FLCs) in this paper. Two types of MFs are considered in the study: triangular and trapezoidal ones. The performance and efficiency of the CSO algorithm are particularly good when perturbations are added during the execution in each control problem. Two benchmark control problems: Water Tank Controller and Inverted Pendulum Controller are considered for testing the proposed approach. Also, the optimal design of a fuzzy controller for trajectory tracking of an Autonomous Mobile Robot (AMR) is considered to test the CSO. The main goal is to highlight the efficiency of CSO algorithm in finding optimal fuzzy controllers of non-linear plants. Two types of perturbations are considered in each control problem. Results show that the CSO algorithm presents excellent results in the field of Fuzzy Logic Controllers. Two types of Fuzzy Inference Systems: Takagi-Sugeno and Mamdani FLSs, are implemented in this paper. The most important metrics usually applied in control are used in this paper, such as: Integral Time Absolute Error (ITAE), Integral Time Squared Error (ITSE), Integral Absolute Error (IAE), Integral Square Error (ISE), Mean Square Error (MSE), and Root Mean Square Error (RMSE).