Open-loop Solution Research Articles

One Class of Stackelberg Linear–Quadratic Differential Games with Cheap Control of a Leader: Asymptotic Analysis of an Open-Loop Solution

We consider a two-player finite horizon linear–quadratic Stackelberg differential game. For this game, we study the case where the control cost of a leader in the cost functionals of both players is small, which means that the game under consideration is a cheap control game. We look for open-loop optimal players’ controls of this game. Using the game’s solvability conditions, the obtaining such controls is reduced to the solution to a proper boundary-value problem. Due to the smallness of the leader’s control cost, this boundary-value problem is singularly perturbed. Asymptotic behavior of the solution to this problem is analyzed. Based on this analysis, the asymptotic behavior of the open-loop optimal players’ controls and the optimal values of the cost functionals is studied. Using these results, asymptotically suboptimal players’ controls are designed. An illustrative example is presented.

Optimal quadratic full state feedback control of nonlinear affine systems

The solution of the optimal quadratic full-state feedback closed-loop control of a nonlinear affine system problem is presented and solved. This solution uses the State-Dependent Coefficient (SDC) form of a nonlinear system and the Calculus of Variations. Further, it is shown that this solution enables the computation of the value function associated with the respective Hamilton-Jacobi-Bellman equation thus satisfying the necessary and sufficient conditions for optimality. The use of the SDC form representation in the optimal solution for a nonlinear affine system results in an explicit and exact full-state feedback closed-loop control implementation as opposed to the existing open-loop solutions of the optimal control. This solution, like for the linear case, is non-causal and thus cannot be solved in real-time. The closed-loop application involves precomputing a time-varying full-state feedback matrix. The optimal feedback is a linear combination of the current state. The time-varying gain matrix is the backward solution of a non-symmetric matrix Riccati equation. Conditions for the stability of the full-state feedback are presented. This new representation of the optimal solution gives a well-understandable algorithm that enables implementation in the closed loop of the optimal feedback control of a non-linear system which is not possible with the existing optimal open loop representations.

Convexified Open-Loop Stochastic Optimal Control for Linear Systems With Log-Concave Disturbances

We consider open-loop solutions to the stochastic optimal control of a linear dynamical system with an additive non-Gaussian, log-concave disturbance. We propose a novel, sampling-free approach, based on characteristic functions and convex optimization, to cast the stochastic optimal control problem as a difference-of-convex program. Our method invokes higher moments, resulting in less conservatism compared to moment-based approaches. We employ piecewise affine approximations and the convex-concave procedure for efficient solution via standard conic solvers. We demonstrate that the proposed solution is competitive with sampling and moment based approaches, without compromising probabilistic constraints.

Sparse feedback controller: from open-loop solution to closed-loop realization

Sparse feedback controller: from open-loop solution to closed-loop realization

LQG Differential Stackelberg Game Under Nested Observation Information Patterns

We investigate the linear quadratic Gaussian Stackelberg game under a class of nested observation information patterns. The follower uses its observation data to design its strategy, whereas the leader implements its strategy using global observation data. We show that the solution requires solving a new type of forward-backward stochastic differential equation, whose drift components contain two conditional expectation terms associated to the adjoint variables. We then propose a method to find the functional relations between each adjoint pair, i.e., each pair formed by an adjoint variable and the conditional expectation of its associated state. The proposed method follows a layered pattern. More precisely, in the inner layer, we seek the functional relation for the adjoint pair under the sigma-sub-algebra generated by follower's observation information; and in the outer layer, we look for the functional relation for the adjoint pair under the sigma-sub-algebra generated by leader's observation information. Our result shows that the optimal open-loop solution admits an explicit feedback type representation.

Recycling glass fibers from thermoset epoxy composites by in situ oxonium-type polyionic liquid formation and naphthalene-containing superplasticizer synthesis with the degradation solution of the epoxy resin

Glass fibre-reinforced thermoset (GFRT) epoxy composites with permanent cross-linked polymer networks are difficult to recycle, leading to the production of large amounts of waste. Developing an efficient approach to reuse both the high-value fibre and the resin matrices is thus imperative. In this work, we explored the application of H2SO4 (98%) to react with ether and benzene groups in the epoxy backbone at room temperature. As a result, a new family of poly(ionic liquid)s (PILs) comprising cationic oxonium was produced. The in situ oxonium PIL formation enables clean glass fibres to be released with well-maintained surface microscopic morphology, chemical structure and tensile modulus close to the virgin glass fibres (vGFs). Furthermore, the dissolved matrix solution functions as a synergistic sulfonation agent in the synthesis of naphthalene containing superplasticizer. The obtained additive is superior to the fresh H2SO4 sulfonated naphthalene formaldehyde condensate (SNF) in reducing water content (33% vs 19%) and improving compressive strength of cementitious materials by 29%. Therefore, this novel eco-friendly strategy provides an open-loop solution for GFRT composites recycling at ambient conditions producing no new waste.

An open-loop solution for a stochastic problem with imperfect state information and chance-constraint adjusted by an optimal gain

This study aims to develop an optimal aggregate production planning policy for a reverse logistics system with imperfect state information and chance constraints. The system consists of a forward channel for storing and distributing products and a backward channel for collecting, remanufacturing, or discarding products. The paper presents a chance-constrained linear-quadratic Gaussian optimization model that considers imperfect state information and formulates an equivalent deterministic problem. It also introduces a minimum variance problem to address the uncontrolled variance of the serviceable inventory variable, whose result is an optimal gain to balance the conditional variances of inventory and production over time, making the solution more cost-efficient. The open-loop solution with the minimum variance gain shows its efficiency through a simple example, reducing the total cost of production by controlling the growth of inventory and production variances. Furthermore, this approach offers a practical way for managers to create inventory-production scenarios for decision-making in a reverse logistics system with imperfect state information and chance constraints.

A Dynamic Hierarchical Framework for IoT-Assisted Digital Twin Synchronization in the Metaverse

Metaverse, also known as the Internet of 3-D worlds, has recently attracted much attention from both academia and industry. Each virtual subworld, operated by a virtual service provider (VSP), provides a type of virtual service. Digital twins (DTs), namely, digital replicas of physical objects, are key enablers. Generally, a DT belongs to the party that develops it and establishes the communication link between the two worlds. However, in an interoperable metaverse, data-like DTs can be “shared” within the platform. Therefore, one set of DTs can be leveraged by multiple VSPs. As the quality of the shared DTs may not always be satisfying, in this article, we propose an agile solution, i.e., a dynamic hierarchical framework, in which a group of Internet of Things devices in the lower level are incentivized to collectively sense physical objects’ status information and VSPs in the upper level determine synchronization intensities to maximize their payoffs. We adopt an evolutionary game approach to model the devices VSP selections and a simultaneous differential game to model the optimal synchronization intensity control problem. We further extend it as a Stackelberg differential game by considering some VSPs to be first movers. We provide open-loop solutions based on the control theory for both formulations. We theoretically and experimentally show the existence, uniqueness, and stability of the equilibrium to the lower level game and further provide a sensitivity analysis for various system parameters. Experiments show that the proposed dynamic hierarchical game outperforms the baseline.

Deterministic dynamic Stackelberg games: Time-consistent open-loop solution

In this paper, the known deterministic linear-quadratic Stackelberg game is revisited, whose open-loop Stackelberg solution actually possesses the nature of time inconsistency. To handle this time inconsistency, a two-tier game framework is introduced: the upper-tier game works according to Stackelberg’s scenario between a leader and a follower, and the outcomes of two lower-tier intertemporal games characterize, respectively, the follower’s and the leader’s response controls that mimic the notion of time-consistent open-loop equilibrium control in existing literature. The obtained open-loop equilibrium of this two-tier game is shown to be weakly time-consistent in the sense that the adopted policies will no longer be denied in the future only if past policies are consistent with the equilibrium policies. On the existence and uniqueness of such a solution, necessary and sufficient conditions are obtained, which are characterized via the solutions of several Riccati-like equations.

Treatise on Analytic Nonlinear Optimal Guidance and Control Amplification of Strictly Analytic (Non-Numerical) Methods.

Optimal control is seen by researchers from a different perspective than that from which the industry practitioners see it. Either type of user can easily become confounded when deciding which manner of optimal control should be used for guidance and control of mechanics. Such optimization methods are useful for autonomous navigation, guidance, and control, but their performance is hampered by noisy multi-sensor technologies and poorly modeled system equations, and real-time on-board utilization is generally computationally burdensome. Some methods proposed here use noisy sensor data to learn the optimal guidance and control solutions in real-time (online), where non-iterative instantiations are preferred to reduce computational burdens. This study aimed to highlight the efficacy and limitations of several common methods for optimizing guidance and control while proposing a few more, where all methods are applied to the full, nonlinear, coupled equations of motion including cross-products of motion from the transport theorem. While the reviewed literature introduces quantitative studies that include parametric uncertainty in nonlinear terms, this article proposes accommodating such uncertainty with time-varying solutions to Hamiltonian systems of equations solved in real-time. Five disparate types of optimum guidance and control algorithms are presented and compared to a classical benchmark. Comparative analysis is based on tracking errors (both states and rates), fuel usage, and computational burden. Real-time optimization with singular switching plus nonlinear transport theorem decoupling is newly introduced and proves superior by matching open-loop solutions to the constrained optimization problem (in terms of state and rate errors and fuel usage), while robustness is validated in the utilization of mixed, noisy state and rate sensors and uniformly varying mass and mass moments of inertia. Compared to benchmark, state-of-the-art methods state tracking errors are reduced one-hundred ten percent. Rate tracking errors are reduced one-hundred thirteen percent. Control utilization (fuel) is reduced eighty-four percent, while computational burden is reduced ten percent, simultaneously, where the proposed methods have no control gains and no linearization.

On the Accuracy of the Model Predictive Control Method

The paper investigates the accuracy of the model predictive control (MPC) method for finding on-line approximate optimal feedback control for Bolza-type problems on a fixed finite horizon. The predictions for the dynamics, the state measurements, and the solution of the auxiliary open-loop control problems that appear at every step of the MPC method may be inaccurate. The main result provides an error estimate of the MPC-generated solution compared with the optimal open-loop solution of the “ideal” problem, where all predictions and measurements are exact. The technique of proving the estimate involves an extension of the notion of strong metric subregularity of set-valued maps and utilization of a specific new metric in the control space, which makes the proof nonstandard. The result is specialized for two problem classes: coercive problems and affine problems.

Time-Fuel Efficient Intermittent Feedback Control of LTI Systems

To derive analytical feedback solution of time- $L_{1}$ optimal control problem is a challenging task. In this regard, this letter proposes an alternative strategy that involves intermittent application of open loop time- $L_{1}$ optimal solution. When disturbances affect the system, such policy is shown to retain the system states to within a small bounded safe region about the origin. The key idea is to switch the processor in between active and idle mode. During the active mode, the states are measured, open-loop optimal control policy is computed and transmitted along the communication link to the actuator. While during the idle mode, the precomputed control profile is applied and rest of the system resources remain inactive. The open loop solution is obtained by solving a set of optimization problems with an additional bound constraint on the update time instances. The bound constraints are derived using an algorithm to ensure that the safe region (defined in terms of attainable set) is always inside the null controllable region or the reachable set. Since attainable and reachable sets are approximated as convex polytopes, the bound constraints are obtained through linear inequalities. Simulation results show that under the proposed control strategy, system achieves practical stability with reduced usage of system resources.

A Payoff Augmentation Approach to Two Pursuers and One Evader Inertial Model Differential Game

In this paper, we study the two-on-one inertial model pursuit evasion differential game with a payoff augmentation approach. In particular, the pursuit evasion game is treated as a deadline game, and its payoff function is augmented to eliminate the deadline as well as to facilitate a more sophisticate model. In addition, we introduce the retrogressive path for analytical solution to the equilibrium strategy for the closed loop game, and proceed to demonstrate that the closed loop game can be converted to a one-side optimization problem for the evader, with the help of the augmented payoff function, under an open loop Stackelberg strategy. More specifically, we establish the conditional equivalency between the open loop solution and the closed loop one. Simulation results verify the effectiveness of the proposed strategy.

Asymptotic analysis and open-loop solutions of one class of partial cheap control zero-sum differential games with state and control delays

Asymptotic analysis and open-loop solutions of one class of partial cheap control zero-sum differential games with state and control delays

Automatic locking of a parametrically resonating, base-excited, nonlinear beam

Described is a closed-loop control scheme capable of stabilizing a parametrically excited nonlinear structure in several vibration modes. By setting the relative phase between the spatially filtered response and the excitation, the open-loop unstable solution branches are stabilized under a 2:1 parametric excitation of a chosen mode of vibration. For a given phase, the closed-loop automatically locks on a limit cycle, through an Autoresonance scheme, at any desired point on the solution branches. Axially driven slender beams and nanowires develop large transverse vibration under suitable amplitudes and frequency base-excitation that are sensitive to small potential coupled field. To utilize such a structure as a sensor, stable and robust operation are made possible by the control scheme. In addition, an optimal operating point with large sensitivity to the sensed potential field can be set using phase as a tunable parameter. Detailed analysis of the dynamical behavior, experimental verifications, and demonstrations sheds light on some features of the system dynamics.

A Novel Wavelet-based Optimal Linear Quadratic Tracker for Time-varying Systems with Multiple Delays

A novel method for solving optimal tracking control of linear quadratic time-varying systems with differentforms of time delays in state and input variables and with constraints is presented in this paper. Using the concepts of two powerful wavelets, Legendre and Chebyshev wavelets, we convert the optimal tracking problem to a static optimization one. The method is presented in a general from by which one can utilize it by other wavelets. The proposed method has the ability to solve the problems with systems of integer and fractional orders. After determining open-loop solutions of time-delay tracking systems, closed-loop suboptimal controller is designed. A highly successful wavelet-based suboptimal controller is introduced in this work. This alternative method is applied on some optimal tracking systems.

Natural Walking With Musculoskeletal Models Using Deep Reinforcement Learning

Human gait optimality has been investigated recently, with the development of detailed musculoskeletal models, through trajectory optimization approaches or deep reinforcement learning (DRL). Trajectory optimization studies are limited by the trajectory length and can only generate open-loop solutions. While existing DRL solutions provide closed-loop control policies without trajectory length limit, they either do not evaluate the naturalness of the behaviour, or directly impose experimental tracking data. In this letter, a DRL-based approach is proposed with a nature-inspired curriculum learning (CL) scheme and a neuromechanically-inspired reward function. This approach generates close-to-natural human walking without the aid of experimental data. Our CL scheme is realized by an evolving reward function, targeting simpler behaviours such as standing and stepping first, then gradually refining the gait. The emerged gait from the closed-loop stochastic policy demonstrated a strong correlation with human gait kinematics, with Pearson correlations of 0.95 and 0.83 at the hip and knee, respectively, and higher gait symmetry than two other DRL-based control policies without CL. Our approach was also found to have efficient convergence to walking-capable policy. This approach can facilitate the development of assistive robotic systems by providing a “human” controller, and could enable decentralized adaptation between the agent and the assistive robotic devices.

ASPEN PLUS desulfurization simulations for the scrubber of a large-scale marine diesel engine: main scrubbing section's desulfurization share optimization and superiority confirmation for the seawater/seawater cascade-scrubbing solution.

Aiming at perfecting the proposed three-stage cascade-scrubbing desulfurization technology, ASPEN PLUS simulations of seawater/seawater cascade-scrubbing desulfurization for an exhaust gas scrubber designed for a 10-MW large-scale marine diesel engine were performed to fix a reasonable desulfurization share for the main scrubbing section. Accordingly, three typical desulfurization share setups of 80%, 85%, and 90% were compared under various operational conditions by varying the fuel-sulfur content, liquid/gas ratio, and seawater alkalinity. In complying with the desulfurization requirements of emission control areas, 85% was obtained as the optimal desulfurization share among the three setups. Again, under the selected 85% setup, the cascade-scrubbing desulfurization was numerically compared with the conventional once-through open-loop solution, so as to confirm the seawater-saving advantage of the cascade-scrubbing desulfurization technology used for the exhaust gas of large-scale marine diesels. At relatively higher sulfur content above 3.5%, the open-loop solution solely achieved the desulfurization requirements of non-emission control areas by using higher liquid/gas ratio levels of 6.5-8.0L/Nm3, while fully failed to meet the much stricter requirements of emission control areas except the lowest 1.5% fuel-sulfur conditions. On the contrary, the cascade-scrubbing solution could meet the requirements of emission control areas under various fuel-sulfur content conditions and reduced the liquid/gas ratio by about 45%, in comparison to the open-loop solution.

Linear-Quadratic Stochastic Stackelberg Differential Games for Jump-Diffusion Systems

This paper considers linear-quadratic (LQ) stochastic leader-follower Stackelberg differential games for jump-diffusion systems with random coefficients. We first solve the LQ problem of the follower using the stochastic maximum principle and obtain the state-feedback representation of the open-loop optimal solution in terms of the integro-stochastic Riccati differential equation (ISRDE), where the state-feedback type control is shown to be optimal via the completion of squares method. Next, we establish the stochastic maximum principle for the indefinite LQ stochastic optimal control problem of the leader using the variational method. However, to obtain the state-feedback representation of the open-loop solution for the leader, there is a technical challenge due to the jump process. To overcome this limitation, we consider two different cases, in which the state-feedback type optimal control for the leader in terms of the ISRDE can be characterized by generalizing the Four-Step Scheme. Finally, in these two cases, we show that the state-feedback representation of the open-loop optimal solutions for the leader and the follower constitutes the Stackelberg equilibrium. Note that the (indefinite) LQ control problem of the leader is new and nontrivial due to the coupled FBSDE constraint induced by the rational behavior of the follower.

Prediction error's minimization through a controller based on a Metaheuristic Algorithm

A Metaheuristic Algorithm (MA) can be a realistic method to solve a given Optimal Control Problem (OCP), but the result is an open-loop solution. If the Metaheuristic Algorithm is integrated within the Model Predictive Control (MPC) structure, a closed-loop solution can be achieved. The controller works using a prediction technique and prediction error's minimization. On the other side, the MA optimizes (minimizes or maximizes) the OCP's objective function. The controller is faced with two optimization tasks. This paper proves through theoretical analysis and simulations that the prediction error's minimization is implicitly accomplished.