Multi-stage Optimal Control Problem Research Articles

An optimal midcourse guidance method for dual pulse air-to-air missiles using linear Gauss pseudospectral model predictive control method

This paper proposes an optimal midcourse guidance method for dual pulse air-to-air missiles, which is based on the framework of the linear Gauss pseudospectral model predictive control method. Firstly, a multistage optimal control problem with unspecified terminal time is formulated. Secondly, the control and terminal time update formulas are derived analytically. In contrast to previous work, the derivation process fully considers the Hamiltonian function corresponding to the unspecified terminal time, which is coupled with control, state, and costate. On the assumption of small perturbation, a special algebraic equation is provided to represent the equivalent optimal condition for the terminal time. Also, using Gauss pseudospectral collocation, error propagation dynamical equations involving the first-order correction term of the terminal time are transformed into a set of algebraic equations. Furthermore, analytical modification formulas can be derived by associating those equations and optimal conditions to eliminate terminal error and approach nonlinear optimal control. Even with their mathematical complexity, these formulas produce more accurate control and terminal time corrections and remove reliance on task-related parameters. Finally, several numerical simulations, comparisons with typical methods, and Monte Carlo simulations have been done to verify its optimality, high convergence rate, great stability and robustness.

Trajectory optimization of rocket recovery based on Neural Network and Genetic Algorithm

After the completion of the launch mission, the soft landing recovery and reuse of the carrier rocket’s first-stage can effectively control the landing area to ensure safety and significantly reduce the cost of space launch transportation. The trajectory optimization of the whole process from the first section to the landing site is significant to fuel saving. In the entire process, the first-stage needs to go through three startup ignitions of course adjustment, power deceleration, and vertical landing. For this segmented fuel optimization problem, optimizing the thrust size and direction during three sections and the switching time between each section is necessary. Optimizing the vertical landing section is essential to ensure the accuracy of landing position and speed, which greatly complicates the whole process’s optimization. Therefore, the optimization process of the vertical landing is substituted by a neural network predictor, which is used to obtain the mapping relationship between the initial state of the vertical landing section and the fuel consumption of the segment. Besides, the thrust profiles of the first two startup sections are reasonably handled based on physical cognition. The combination of convex optimization and neural network successfully converted the multi-stage optimal control problem into a parameter optimization problem and solved it by a genetic algorithm. Optimization results were compared with the conventional method, which indicated its superiority.

A Learning Based Intelligent Train Regulation Method With Dynamic Prediction for the Metro Passenger Flow

With the acceleration of urbanization, the dynamic passenger flow has an ever-growing impact on the actual train operation. In this paper, we propose a learning based intelligent train regulation method with dynamic passenger flow prediction. To capture the characteristics of the dynamic metro passenger flow, a convolutional neural network is established to predict the real-time passenger flow from two dimensions including space and time. As the prediction accuracy is restricted by the insufficiency of the practical passenger flow data, a deep convolutional generative adversarial network is constructed to generate data that have the same distribution as the original passenger flow dataset. Then, by considering the effects of the dynamic passenger flow on the train operation and the train capacity constraints, the dynamic train regulation is formulated as a multi-stage optimal control problem with the objective function of minimizing the train traction energy consumption and the total traveling time of passengers. To efficiently obtain the optimal regulation strategy at each decision step, a deep Q-network algorithm is proposed to solve the formulated problem such the dimensionality curse caused by the excessive state space is avoided. The numerical experiments demonstrate the high efficiency and effectiveness of our proposed algorithm and model.

Data-Driven Iterative Optimal Control for Switched Dynamical Systems

This article presents a data-driven algorithm to compute optimal control inputs for input-constrained nonlinear optimal control problems with switched dynamics. We consider multi-stage optimal control problems where the control inputs and the switching instants are both unknown. Our key contribution is the new iterative online optimal control algorithm which mitigates sub-optimal control caused by model bias in the challenging class of under-actuated and intrinsically unstable switched dynamical systems. This is achieved by estimating the cost and computing the control inputs along measured trajectories of the controlled system instead of doing the same procedure along error-prone trajectories predicted by an inexact model. The algorithm is evaluated using an under-actuated and intrinsically unstable hopping robot in a simulation environment. The algorithm enables real-time data-driven optimal control using inaccurate models.

Risk-Averse Optimal Control in Continuous Time by Nesting Risk Measures

This paper extends dynamic control problems from a risk-neutral to a risk-averse setting. We establish a limit for consecutive risk-averse decision making by consistently and adequately nesting coherent risk measures. This approach provides a new perspective on multistage optimal control problems in continuous time. For the limiting case, we elaborate a new dynamic programming principle, which is risk averse, and give risk-averse Hamilton–Jacobi–Bellman equations by generalizing the infinitesimal generator. In doing so, we provide a constructive explanation of the driver g in g-expectation, a dynamic risk measure based on backward stochastic differential equations. Funding: This work was supported by Deutsche Forschungsgemeinschaft [Project-ID 416228727 – SFB 1410].

Convexity and Feedback in Approximate Dynamic Programming for Delivery Time Slot Pricing

We consider the revenue management problem of finding profit-maximizing prices for delivery time slots in the context of attended home delivery. This multistage optimal control problem admits a dynamic programming (DP) formulation that is intractable for realistic problem sizes due to the so-called “curse of dimensionality.” We therefore study three approximate DP algorithms both from a numerical and control-theoretical perspective. Our analysis is based on real-world data, from which we generate multiple scenarios to stress-test the robustness of the pricing policies to errors in model parameter estimates. Our theoretical analysis and numerical benchmark tests indicate that one of these algorithms, namely gradient-bounded DP, dominates the others with respect to computation time and profit-generation capabilities of the pricing policies that it generates.

A Note on Liao’s Recurrent Neural-Network Learning for Discrete Multi-stage Optimal Control Problems

The roots of neural-network backpropagation (BP) may be traced back to classical optimal-control gradient procedures developed in early 1960s. Hence, BP can directly apply to a general discrete N-stage optimal control problem that consists of N stage costs plus a terminal state cost. In this journal (Liao in Neural Process Lett 10:195–200, 1999), given such a multi-stage optimal control problem, Liao has turned it into a problem involving a terminal state cost only (via classical transformation), and then claimed that BP on the transformed problem leads to new recurrent neural network learning. The purpose of this paper is three-fold: First, the classical terminal-cost transformation yields no particular benefit for BP. Second, two simulation examples (with and without time lag) demonstrated by Liao can be regarded naturally as deep feed-forward neural-network learning rather than as recurrent neural-network learning from the perspective of classical optimal-control gradient methods. Third, BP can readily deal with a general history-dependent optimal control problem (e.g., involving time-lagged state and control variables) owing to Dreyfus’s 1973 extension of BP. Throughout the paper, we highlight systematic BP derivations by employing the recurrence relation of nominal cost-to-go action-value functions based on the stage-wise concept of dynamic programming.

Heat exchanger network cleaning scheduling: From optimal control to mixed-Integer decision making

An approach for optimising the cleaning schedule in heat exchanger networks (HENs) subject to fouling is presented. This work focuses on HEN applications in crude oil preheat trains located in refineries. Previous approaches have focused on using mixed-integer nonlinear programming (MINLP) methods involving binary decision variables describing when and which unit to clean in a multi-period formulation. This work is based on the discovery that the HEN cleaning scheduling problem is in actuality a multistage optimal control problem (OCP), and further that cleaning actions are the controls which appear linearly in the system equations. The key feature is that these problems exhibit bang-bang behaviour, obviating the need for combinatorial optimisation methods. Several case studies are considered; ranging from a single unit up to 25 units. Results show that the feasible path approach adopted is stable and efficient in comparison to classical methods which sometimes suffer from failure in convergence.

Exploitation of the control switching structure in multi-stage optimal control problems by adaptive shooting methods

The adaptive switching structure approach is generalized from single-stage problems and single-shooting to multi-stage problems and multiple-shooting. This generalization is based on previous work on the exploitation and detection of the control switching structure and wavelet-based control grid refinement for single-stage problems. Here, single-shooting is employed to transcribe the multi-stage optimal control problem (OCP) into a nonlinear programming problem. The proposed multi-stage formulation is also capable to represent the transcription of single-stage OCP stemming from multiple-shooting. Thus, the previously reported adaptive multiple-shooting approach is extended by an adaptation of the switching structure. Finally, a new stopping criterion is introduced that measures the intermediate constraint violation at the optimal solution.The proposed adaptive switching structure detection is illustrated for a multi-stage and a multiple-shooting problem using the Williams–Otto semi-batch reactor. A solution of user-specified accuracy in the objective and the path-constraints can be obtained using only few decision variables.

Aircraft optimal target aiming control based on Gauss pseudospectral method

In order to realize aircraft optimal target aiming in the situation of combat duel, a control method based on Gauss Pseudospectral Method( GPM) was proposed. Taking agility and multi-constraint into consideration, the dynamic equation of the aircraft was modeled, the two-stage target aiming condition expression was deduced, and the optimal index was designed. Afterwards, the aircraft optimal aiming control was described as the multi-stage optimal control problem with constraint and unknown final time. The GPM was used to equally convert the continuous optimal boundary value problem to a discrete Nonlinear Programming( NLP) problem and the initial solution was preprocessed through Genetic Algorithm( GA),then, the Sequential Quadratic Programming( SQP) algorithm was applied to solve it. The simulation results show that it can realize target aiming effectively and satisfy weapon launch condition.

Capital Accumulation and Embodied Technological Progress

This paper combines technology adoption with capital accumulation taking into account technological progress. We model this as a multi-stage optimal control problem and solve it using the corresponding maximum principle. The model with linear revenue can be solved analytically, while the model with market power is solved numerically. We obtain that investment jumps upwards right at the moment that a new technology is adopted. We find that, if the firm has market power, the firm cuts down on investment before a new technology is adopted. Furthermore, we find that larger firms adopt a new technology later.

An efficient multiple shooting based reduced SQP strategy for large-scale dynamic process optimization. Part 1: theoretical aspects

Optimal design and operation of complex chemical processes often require the solution of intricate dynamic optimization problems. A tailored simultaneous solution strategy based on multiple shooting and reduced SQP is presented. This reduced-space boundary value problem (BVP) approach allows an efficient and robust solution of multistage optimal control and design optimization problems for large, sparse DAE process models of index one. The current paper describes the theoretical aspects of the method. Utilizing the natural decomposition of the states into differential and algebraic variables, the structured NLP problem which results from the multiple shooting discretization of the optimization BVP is projected onto the reduced space of differential variables and control parameters. It is shown that this projection can be obtained very efficiently through direct computation of the reduced linearized constraint system via directional sensitivities. Like the original full-space BVP approach, the reduced-space formulation lends itself well to parallel computation. An implementation of the new strategy is provided by the modular optimal control package MUSCOD-II. Software aspects and applications are discussed in a second paper (Part II Software Aspects and Applications, 2002).

Optimal control laws for lot-sizing and timing of jobs on a single production facility

The optimal control of a single machine processing a certain number of jobs and modeled as a discrete-event dynamic system is considered. The number of jobs and their sequence are fixed, whereas their timing and sizes represent the control variables of the system. The objective function to be optimized is a weighted sum of the quadratic earliness and tardiness of each job, and of the quadratic deviations of job lot sizes and actual machine service speeds from those specified by the production demand and by the regular machine speeds. An optimization problem with quadratic cost function and nonlinear constraints is stated and formalized as a multistage optimal control problem. Necessary conditions to be satisfied by an optimal control sequence are derived. A simpler model is also considered in which the machine speeds are fixed; in this case, the control problem is solved by a procedure making use of dynamic programming techniques. The optimal control laws at each stage are thus obtained.

Necessary conditions for infinite-horizon discounted two-stage optimal control problems

In many situations in Economics, one would like to analyse the optimal timing of switching between alternative and consecutive regimes. A natural framework for such an analysis seems to be one of multi-stage optimal control problems, where the switching instants between regimes are endogenously determined. The existing literature considers only the case of two-stage optimal control problems with finite horizon. However, in many cases, it seems more appropriate to consider a problem with an infinite horizon. By deriving the appropriate necessary conditions for such a problem we generalise the methodology in analysing endogenous regime switches.

Synthesis of batch reaction/distillation processes using detailed dynamic models

This paper considers the synthesis of processes comprising batch reaction and distillation tasks. Complex process features, such as material recycles and shared intermediates are taken into account. The problem is formulated as a multistage optimal control problem. A control vector parametrisation approach is utilised to reduce this infinite dimensional dynamic problem to a finite dimensional nonlinear programming problem. An example involving an exothermic reaction step followed by a batch distillation is presented to illustrate the proposed approach.

Optimization of discrete charge batch reactors

A special class of multistage optimal control problems arises in the case of batch reactors fed by discrete instantaneous additions of raw material. The underlying differential-algebraic equations (DAEs) remain unchanged throughout the time horizon of interest, but the instantaneous additions of material are impulsive inputs that result in discontinuities in the state variables. Optimization parameters can include, among others, the amount of each charge and the reaction stage duration following each addition. In this paper, the problem of optimizing the operation of discrete charge batch reactors is addressed within a general framework that is independent of kinetic mechanisms and nonidealities. No approximation need be involved as the solution procedure is based on a general purpose robust algorithm for dynamic optimization problems.

A projected Newton method for minimization problems with nonlinear inequality constraints

Recently developed projected Newton methods for minimization problems in polyhedrons and Cartesian products of Euclidean balls are extended here to general convex feasible sets defined by finitely many smooth nonlinear inequalities. Iterate sequences generated by this scheme are shown to be locally superlinearly convergent to nonsingular extremals $$\bar u$$ , and more specifically, to local minimizers $$\bar u$$ satisfying the standard second order Kuhn-Tucker sufficient conditions; moreover, all such convergent iterate sequences eventually enter and remain within the smooth manifold defined by the active constraints at $$\bar u$$ . Implementation issues are considered for large scale specially structured nonlinear programs, and in particular, for multistage discrete-time optimal control problems; in the latter case, overall per iteration computational costs will typically increase only linearly with the number of stages. Sample calculations are presented for nonlinear programs in a right circular cylinder in ?3.