Control Problems Research Articles

Optimal bipartite consensus control for heterogeneous unknown multi-agent systems via reinforcement learning

This study focuses on addressing optimal bipartite consensus control (OBCC) problems in heterogeneous multi-agent systems (MASs) without relying on the agents' dynamics. Motivated by the need for model-free and optimal consensus control in complex MASs, a novel distributed scheme utilizing reinforcement learning (RL) is proposed to overcome these challenges. The MAS network is randomly partitioned into sub-networks where agents collaborate within each subgroup to attain tracking control and ensure convergence of positions and speeds to a common value. However, agents from distinct subgroups compete to achieve diverse tracking objectives. Furthermore, the heterogeneous MASs considered have unknown first and second-order dynamics, adding to the complexity of the problem. To address the OBCC issue, the policy iteration (PI) algorithm is used to acquire solutions for discrete-time Hamilton-Jacobi-Bellman (HJB) equations while implementing a data-driven actor-critic neural network (ACNN) framework. Ultimately, the accuracy of our proposed approach is confirmed through the presentation of numerical simulations.

A review of dynamic optimization in aquaculture production economics

AbstractDynamic optimization (DO) has been applied to aquaculture industry to determine optimal management strategies of aquaculture production systems. Optimal control of feeding, optimal stocking successions, and harvesting time among other factors has been studied. This work shows a review of the application of optimal control theory (OCT) and dynamic programming in economics and management of aquaculture. The Pontriagyn's maximum principle and the Bellman's optimality are the most commonly used mechanisms to solve control problems that optimize the producer's benefit or cost. Recently, model‐based predictive control has also been applied. Such tools have been used in aspects involving the design, planning, and monitoring of variables relevant to the optimal management of the culture system. It was found that in aquaculture there is low scientific productivity in the application of DO in bioeconomic models. In this review, an example of OCT applied to the control of feeding of farmed tilapia using a bioeconomic model is shown. DO represents a useful tool for optimal decision‐making and this review discusses the implications of the use of DO in aquaculture and recommendations for its use in the future.

A gradient-descent iterative learning control algorithm for a non-linear system

Original iterative learning control (OILC) has been proved a powerful tool in dealing with the model-free control problems by repetitively corrections based on the control error. However, the steady-state error under widely-used proportional-type original iterative learning control (P-type OILC) is highly corresponded to the proportional learning gain, making the algorithm parameter-determined. Therefore, a new gradient-descent iterative learning control (GDILC) algorithm is proposed to achieve a parameter-free approach by simulating the gradient-descent process. First, GDILC problem is formulated mathematically. Next, the idea of the algorithm is proposed, the analyses of the convergence and the steady-state error are conducted and the algorithm is implemented. GDILC will generate a random correction with a gradient-descent upper bound, rather than a correction proportional to the error in P-type OILC. Finally, illustrative and application simulations are conducted to validate the algorithm. Results show that the algorithm will be convergent after adequate iterations under proper corrections. The steady-state error will be less affected by the algorithm parameters under GDILC than that under OILC.

Sliding dynamics of a Filippov ecological system with nonlinear threshold control and pest resistance

By applying the integrated pest management strategy to address pest control problems, Filippov pest-natural enemy systems have been extensively studied. However, little attention has been paid to pest changing rate and pest resistance in these studies. Therefore, this paper proposes a complex and more realistic Filippov pest-natural enemy system with curved discontinuity boundary by considering Beddington–DeAngelis functional response and pest resistance, as well as incorporating pest changing rate in the threshold control index. Based on the relevant theory of the Filippov system, we analyze a sextic equation and obtain that the proposed Filippov system can have at most six sliding segments. By using qualitative techniques, we discuss the sliding segment and pseudo-equilibrium bifurcations. Numerically, we investigate the local and global sliding bifurcations. It can be detected that one new local sliding bifurcation occurs, which could be called the pseudo-Hopf bifurcation but differs from the traditional pseudo-Hopf bifurcation as it involves two sliding segments with opposite stabilities and generates a sliding limit cycle. We also discover two interesting global sliding bifurcations with novel bifurcation processes, which may be termed triple limit cycle bifurcations as they both relate to the appearance of three nested limit cycles.

Various duality models associated with some constrained robust nonlinear optimal control problems

This paper deals with various duality results for a new class of constrained robust nonlinear control problems. Concretely, we introduce a minimisation problem associated with a vector of functionals of multiple integral type. In addition, the considered multi-objective controlled optimisation problem is governed by several mixed constraints generated by partial derivatives of second order and uncertain data. For this new class of control problems, we establish and characterise Wolfe, Mond-Weir and mixed type robust dual models. To this aim, by using the notion of convex multiple integral functional, generated by a controlled second-order Lagrangian with uncertain data, and the concept of robust weak efficient solution to the problem under study, we build a new mathematical context to state and prove the duality theorems.

Stability for Parametric Control Problems of PDEs via Generalized Differentiation

Stability for Parametric Control Problems of PDEs via Generalized Differentiation

A unified controller of global trajectory tracking and posture regulation for a car‐like mobile robot

SummaryThis work proposes a smooth time‐varying controller for trajectory tracking and posture regulation problems of car‐like mobile robots (CLMR). Currently, few studies focus on the control problems of CLMR's kinematic model. Global trajectory tracking or global posture stabilization controller for CLMR has not yet been proposed. In this study, we propose a global trajectory tracking controller for CLMR for the first time. By setting a specific reference trajectory, the controller is generalized to the posture stabilization task. Obstacle avoidance is also taken into consideration in posture stabilization task. Unlike the prototypical method of transforming the model into a nonholonomic chained‐form system, the proposed method is designed based on the original tracking error equation. Therefore our approach does not have singularities and has a global attraction region. Furthermore, the global convergence of the proposed controllers is strictly proved using the proof by contradiction and Barbalat's lemma. Finally, simulated and actual experiments on a car‐like robot demonstrate the effectiveness of the proposed control scheme.

Comparative analysis of grid-interactive building control algorithms: From model-based to learning-based approaches

Grid-interactive building control poses a critical challenge in the context of grid modernization and decarbonization. Recently, various artificial intelligence-based optimal control approaches have been proposed, providing innovative solutions to optimal building control problems. However, researchers and practitioners are now confronted with a dilemma when selecting appropriate control strategies, ranging from model-based (knowledge-incorporating) to learning-based (data-driven) approaches, and hybrid methods combining them. Although each algorithm in existing literature claims superiority over specific baselines, their performance has never been systematically compared and analyzed, owing to the absence of a unified platform for comprehensive evaluation. To fill this knowledge gap, we identify and implement all state-of-the-art approaches within a modular training and evaluation framework, assessing their efficacy in a grid-interactive building control problem. In this paper, we also introduce a streamlined hybrid method that complements existing hybrid approaches. Our comparative study reveals and quantifies the advantages of hybrid methods: on average, they achieve near-optimal control while requiring less than 14% of the online computation of traditional model-based methods. To achieve this performance, they need as few as 2% of the training samples of purely learning-based methods. Finally, we provide insights into the merits, limitations, and implementation of each method to help researchers better understand the state of the art and future directions. The software framework implemented in this study is open-sourced to facilitate future research.

On the issue of control of neutral-type systems by hybrid controllers with state feedback

For linear autonomous systems of neutral type, an approach to control design problems in the form of feedback based on a new class of hybrid controllers is proposed. The structure of hybrid regulators necessarily includes a differential equation, so the closed system becomes differential-algebraic. A distinctive feature of hybrid controllers is the existence of elementary transformation of equations of a closed-loop system, which make it possible to obtain an independent subsystem of neutral type. In this case, the specified subsystem of neutral type uniquely determines the behavior of the solution $x(t)$ of the original open-loop system (possibly as a vector component of the solution vector of the closed-loop system). The main advantages of using hybrid controllers include the possibility of their application to systems that do not satisfy the “traditional” controllability properties. The properties of hybrid controllers have been studied. The author gives an example of using these controllers to solve a new problem of controlling the coefficients of a characteristic quasi-polynomial in the case where the approaches known in the literature are not applicable. The possibility of using hybrid controllers to solve the $0$-controllability problem using the finite control method is demonstrated.

Reinforcement Learning for Efficient Power Systems Planning: A Review of Operational and Expansion Strategies

The efficient planning of electric power systems is essential to meet both the current and future energy demands. In this context, reinforcement learning (RL) has emerged as a promising tool for control problems modeled as Markov decision processes (MDPs). Recently, its application has been extended to the planning and operation of power systems. This study provides a systematic review of advances in the application of RL and deep reinforcement learning (DRL) in this field. The problems are classified into two main categories: Operation planning including optimal power flow (OPF), economic dispatch (ED), and unit commitment (UC) and expansion planning, focusing on transmission network expansion planning (TNEP) and distribution network expansion planning (DNEP). The theoretical foundations of RL and DRL are explored, followed by a detailed analysis of their implementation in each planning area. This includes the identification of learning algorithms, function approximators, action policies, agent types, performance metrics, reward functions, and pertinent case studies. Our review reveals that RL and DRL algorithms outperform conventional methods, especially in terms of efficiency in computational time. These results highlight the transformative potential of RL and DRL in addressing complex challenges within power systems.

Mixed virtual element methods for elliptic optimal control problems with boundary observations in L2(Γ)

In this paper, we study the mixed virtual element approximation to an elliptic optimal control problem with boundary observations. The objective functional of this type of optimal control problem contains the outward normal derivatives of the state variable on the boundary, which reduces the regularity of solutions to the optimal control problems. We construct the mixed virtual element discrete scheme and derive an a priori error estimate for the optimal control problem based on the variational discretization for the control variable. Numerical experiments are carried out on different meshes to support our theoretical findings.

The pulmonary neuroepithelial body-vagal sensory nerve axis is altered in a mouse model of bronchopulmonary dysplasia (BPD)

Breathing control problems, like apnea of prematurity, are a common complication in preterm infants with bronchopulmonary dysplasia (BPD), but the underlying factors remain incompletely understood. A subpopulation of pulmonary vagal afferent nerve fibers (Calb1+) innervate CO2 and O2 sensitive clusters of lung neuroendocrine cells (neuroepithelial bodies (NEBs)) located along conducting airways. Calb1+ fibers provide direct sensory input to the brainstem that putatively limit tidal volume. NEBs are hyperplastic in BPD infants and may increase activity of inhibitory Calb1+ vagal fibers. Herein, we sought to determine if the NEB-Calb1 lung-brainstem axis is altered in association with breathing control in BPD. BPD in mice was established by exposure to chronic neonatal (postnatal day (P) 0-14) hyperoxia (Hx; 70% O2; n=20) or normoxia (Nx; n=17). Consistent with human BPD, Hx caused alveolar simplification supported by a reduction in alveolar density and alveoli (p<0.05; separate t-tests) in P22 H&E-stained lung sections (n=5-7/group). Like human infants at risk for BPD, Hx mice adapted to increased dead space (caused by reduction in alveoli) by hyperventilating (p<0.05; generalized linear modeling accounting for fat free mass and heat production, measured by NMR and respirometry; n=10-24/group). This was facilitated by an increase in tidal volume that was significantly reduced at P14 but increased at P17 and P21 (n=19-28) (Tukey’s post-hoc: p<0.05). NEBs and associated innervation (vagal and dorsal root ganglia) were next assessed in subsets of mice. First, IT instillation of WGA-594 lectin into subsets of mice (n=9 mice/18 ganglia per group) and subsequent quantification of 594+ neurons per vagal ganglia revealed fewer neurons (p=0.07; t-test) in Hx mice (110.8 ± 6.9 vs 95.2 ± 4.9). Second, CGRP+ innervation to NEBs is reduced in Hx mice (Quantification of CGRP+ nerve fibers only: 17740.4 +/- 4126 vs 10474.6 +/- 2140 +CGRP pixels; p=0.12 n=5-7/group). Third, there was greater frequency of NEBs per unit airway (p<0.05; 0.306 ± 0.03 vs 0.401 ± 0.03) with a tendency for larger NEBs (p=0.06; 9.14 ± 0.7 vs 11.93 ± 1.3) indicating NEB hyperplasia. Together, these data indicate individual NEB innervation is reduced in BPD but that total NEB innervation may remain unchanged. If this is true for Calb1+ vagal fibers remain untested. To functionally assess Calb1-vagal fibers we activated, with DCZ, the excitatory synthetic Gq-DREAD receptor (DTR) expressed in Calb1cre mice via microinjection of AAV-Cre-Gq-mCherry into the left vagal ganglia. DCZ but not vehicle caused a major increase in peak inspiratory flow during acute hypoxic challenges compared to vehicle treated mice (Pre-DMSO, Post-DMSO, Pre-DCZ = ~1ml/sec vs 4ml/sec post-DCZ; p<0.05, n=4/group). Collection of vagal ganglia confirms DTR expression but spillover of DTR is present. Repeating these studies with diluted AAV-Cre-Gq in control and BPD mice are ongoing. Together, our data verify that hyperoxia induced BPD in mice causes a similar reduction in alveoli and compensation to increased dead space by hyperventilation as observed in human BPD. Additionally, BPD structurally alters the NEB-vagal axis but whether these changes are functionally significant to nCOB is being addressed with ongoing studies. AHA 20CDA35310121, Advancing Healthier Wisconsin Endowment. This is the full abstract presented at the American Physiology Summit 2024 meeting and is only available in HTML format. There are no additional versions or additional content available for this abstract. Physiology was not involved in the peer review process.

Flying qualities based time-varying stability augmentation system design for tiltrotor conversion control

Tiltrotors have three flight modes that pose control problems and quality defects during the conversion process. To address this, a novel flying qualities-based time-varying stability augmentation system is designed to achieve multi-mode, nonlinear, and time-varying stability. The system integrates a nonlinear time-varying control law with the flying qualities requirements for all three flight modes. It consists of an inner and outer loop control framework, where the control law in the inner loop is designed based on the Lyapunov theorem of stability. The reference models in the outer loop are derived from the flying qualities criteria to meet level one flying qualities requirements. To evaluate the conversion process, a time-varying flying qualities evaluation method is developed, which includes the conversion path, pilot model, and time-varying flying qualities index. The proposed time-varying stability augmentation control system is then tested through simulation during the conversion process. A pilot-aircraft closed-loop system is established for conducting experiments. Comparison between simulation results and pilot-in-loop experiment results demonstrates the effectiveness of the proposed control system. Furthermore, it proves that the evaluation method is suitable for analyzing time-varying systems. This research can be valuable in designing and evaluating stability augmentation controls for strongly time-varying systems.

Event-Triggered Constrained Optimal Control for Organic Rankine Cycle Systems via Safe Reinforcement Learning.

The organic Rankine cycle (ORC) is an effective application for converting low-grade heat sources into power and is crucial for environmentally friendly production and energy recovery. However, the inherent complexity of the mechanism, its strong and unidentified nonlinearity, and the presence of control constraints severely impair the design of its optimal controller. To solve these issues, this study provides a novel event-triggered (ET) constrained optimal control approach for the ORC systems based on a safe reinforcement learning technique to find the optimal control law. Instead of employing the usual non-quadratic integral form to solve the control-limited optimal control problems, a constraint handling strategy based on a relaxed weighted barrier function (BF) technique is proposed. By adding the BF terms to the original value function, a modified value iteration algorithm is developed to make the control input solutions that tend to violate the constraints be pushed back and maintained in their safe sets. In addition, the ET mechanism proposed in this article is critically required for the ORC systems, and it can significantly reduce the computational load. The combination of these two techniques allows the ORC systems to achieve set-point tracking control and satisfy the control restrictions. The proposed approach is conducted based on a heuristic dynamic programming framework with three neural networks (NNs) involved. The safety and convergence of the proposed approach and the stability of the closed-loop system are analyzed. Simulation results and comparisons are presented to demonstrate its effectiveness.

Distributed event-triggered formation control of UGV-UAV heterogeneous multi-agent systems for ground-air cooperation

Within the context of ground-air cooperation, the distributed formation trajectory tracking control problems for the Heterogeneous Multi-Agent Systems (HMASs) is studied. First, considering external disturbances and model uncertainties, a graph theory-based formation control protocol is designed for the HMASs consisting of Unmanned Aerial Vehicles (UAVs) and Unmanned Ground Vehicles (UGVs). Subsequently, a formation trajectory tracking control strategy employing adaptive Fractional-Order Sliding Mode Control (FOSMC) method is developed, and a Feedback Multilayer Fuzzy Neural Network (FMFNN) is introduced to estimate the lumped uncertainties. This approach empowers HMASs to adaptively follow the expected trajectory and adopt the designated formation configuration, even in the presence of various uncertainties. Additionally, an event-triggered mechanism is incorporated into the controller to reduce the update frequency of the controller and minimize the communication exchange among the agents, and the absence of Zeno behavior is rigorously demonstrated by an integral inequality analysis. Finally, to confirm the effectiveness of the proposed formation control protocol, some numerical simulations are presented.

Fourier–Gegenbauer pseudospectral method for solving periodic fractional optimal control problems

This paper introduces a new, highly accurate model for periodic fractional optimal control problems (PFOCPs) based on Riemann–Liouville and Caputo fractional derivatives (FDs) with sliding fixed memory lengths. This new model enables periodic solutions in fractional-order control models by preserving the periodicity of the fractional derivatives. Additionally, we develop a novel numerical solution method based on Fourier and Gegenbauer pseudospectral approaches. Our method simplifies PFOCPs into constrained nonlinear programming problems (NLPs), readily solvable by standard NLP solvers. Some key innovations of this study include: (i) The derivation of a simplified FD calculation formula to transform periodic FD calculations into evaluations of trigonometric Lagrange interpolating polynomial integrals, efficiently computed using Gegenbauer quadratures. (ii) The introduction of the αth-order fractional integration matrix based on Fourier and Gegenbauer pseudospectral approximations, which proves to be highly effective for computing periodic FDs. A priori error analysis is provided to predict the accuracy of the Fourier–Gegenbauer-based FD approximations. Benchmark PFOCP results validate the performance of the proposed pseudospectral method.

On the Connection Between Costate and the Annihilator of the Hamiltonian Vector Field in Optimal Control Problems

On the Connection Between Costate and the Annihilator of the Hamiltonian Vector Field in Optimal Control Problems

AI‐Based Ensemble Flood Forecasts and Its Implementation in Multi‐Objective Robust Optimization Operation for Reservoir Flood Control

AbstractInforming reservoirs with forecasts is highly important for real‐time flood control. This study proposed a forecast‐informed methodology framework for reservoir flood control operation under uncertainty. A new combination of two post‐processing methods, that is, the Cloud model and error‐based copula functions, were developed to merge individual AI‐based forecasts to ensemble flood forecasts, so called stochastic errors‐based Cloud (SE‐Cloud). A multi‐objective robust optimization model (MRO) integrating the risk, resilience, and vulnerability was then proposed to tackle flood control problems under ensemble forecasts; for comparison, a two‐objective stochastic optimization model (TSO) was developed to minimize the expected highest reservoir level and peak release. The proposed methodology was applied to the Lishimen reservoir in the Shifeng River subbasin, China, aiming to comprehensively verify the relationships among deterministic forecasts, ensemble forecasts, and flood control performance. Results showed that the Cloud model could effectively integrate different models and improve forecast accuracy. But a higher deterministic forecast quality did not consistently result in improved flood control performance. SE‐Cloud could capture the peak flow and effectively characterize forecast uncertainties and increased hypervolume values by 13.14%–39.65% compared to the Cloud model, indicating the superiority of ensemble forecasts in generating robust solutions over individual deterministic forecasts. MRO released more inflow than TSO, decreasing the expected highest water level by 0.05 m and incrementing the expected peak release by 4.29%. However, with downstream resilience value remaining at zero, it is demonstrated that MRO improving upstream vulnerability did not necessarily diminish resilience. The enhanced robustness highlights the potential of AI‐based ensemble forecasts in flood control.

Role of virtual reality in examining the effect of fear of falling (FOF) on postural stability in individuals without and with Parkinson’s disease in Egypt: a mixed-methods feasibility study protocol

BackgroundFalls are common in older people and individuals with neurological conditions. Parkinson’s disease (PD) is known for postural instability causing mobility disabilities, falls and reduced quality of life. The fear...

Design and verification of controller for a small coaxial dual-rotor aircraft

In recent years, coaxial dual-rotor aircraft has attracted much attention due to its unique aerodynamic layout. However, it is difficult to miniaturize the coaxial dual-rotor aircraft that relies on the principle of periodic pitch variation due to complex structure and control difficulties. This study proposes a small cylindrical coaxial dual-rotor aircraft based on a vector motor seat. For the control problems of such coaxial dual-rotor aircraft, a position and attitude controller based on neural networks and adaptive cascaded PID control is designed. A control system model was established and tested using MATLAB/Simulink. The proposed control scheme’s soundness and efficiency were confirmed by creating a prototype and performing flight trials. The results show that compared with the classic PID controller, the control effect of the controller has improved by 36.3% to 54.6%. In the centripetal rotation flight simulation experiment, the displacement error does not exceed 0.2 m, the speed error does not exceed 0.05 m/s, and the attitude angle error does not exceed 0.01 rad. This validates the effectiveness of the designed controller, indicating that the controller can enhance the stability of the system and achieve stable hovering and flight.