Optimal Control Solver Research Articles

Convex Approach to Real-Time Multiphase Trajectory Optimization for Urban Air Mobility

Electric vertical takeoff and landing technology has emerged as a promising solution to alleviate ground transportation congestion. However, the limited capacity of onboard batteries enforces hard time constraints for vehicle operations in a complex urban environment. Therefore, a constrained, real-time trajectory optimization method is necessary to enable safe and energy-efficient vehicle flight. Unfortunately, most existing trajectory optimization methods suffer from low computational efficiency, unpredictable convergence processes, and strong dependence on a good initial trajectory. In this paper, we employ a successive convex programming approach to solve the multiphase minimum-energy-cost electric vertical takeoff and landing vehicle trajectory optimization problem for urban air mobility applications. The main contribution is the development and validation of two novel convexification methods that find approximate optimal solutions to the multiphase nonlinear trajectory optimization problem in real time by solving a sequence of convex subproblems. Specifically, the first method transforms the original nonlinear problem into a sequence of second-order cone programming problems through a convenient change of variables and lossless convexification, while the second approach achieves a similar goal without the need for control convexification. The resulting convex subproblems can be solved reliably in real time by advanced interior point methods. The proposed methods are demonstrated through numerical simulations of two different urban air mobility scenarios and compared with the solution obtained from GPOPS-II, a state-of-the-art general-purpose optimal control solver. Our proposed sequential convex programming methods can obtain near-optimal solutions with faster computational speed than GPOPS-II.

A multi-objective optimization framework for determining optimal chemotherapy dosing and treatment duration

Traditional randomized clinical trials are regarded as the gold standard for assessing the efficacy of chemotherapy. However, this procedure has drawbacks such as high cost, time consumption, and limited patient exploration of treatment regimens. We develop a multi-objective optimization-based framework to address these limitations and determine the best chemotherapy dosing and treatment duration. The proposed framework uses patient-specific biological parameters to create a mathematical model of cell population dynamics in the patient’s body. The framework employs evolutionary heuristic search methods (simulated annealing and genetic algorithms) and a prescriptive analytics approach to optimize therapy sessions that transition from treatment to relaxation. We carefully adjust the chemotherapy dose during treatment to reduce tumor cells while preserving host cells (such as effector-immune cells). We strategically time the relaxation sessions to aid recovery, considering the ability of tumors and healthy cells to regenerate. We use a combined optimization method to determine the length of the session and the amount of drug to be administered. We compare quadratic and linear optimal control solvers for drug administration while genetic algorithms and simulated annealing are used to optimize session length. This approach is especially important in limited healthcare resources, ensuring efficient allocation while accurately identifying high-risk patients to optimize resource allocation and utilization.

Offline supervised learning v.s. online direct policy optimization: A comparative study and a unified training paradigm for neural network-based optimal feedback control

This work is concerned with solving neural network-based feedback controllers efficiently for optimal control problems. We first conduct a comparative study of two prevalent approaches: offline supervised learning and online direct policy optimization. Albeit the training part of the supervised learning approach is relatively easy, the success of the method heavily depends on the optimal control dataset generated by open-loop optimal control solvers. In contrast, direct policy optimization turns the optimal control problem into an optimization problem directly without any requirement of pre-computing, but the dynamics-related objective can be hard to optimize when the problem is complicated. Our results underscore the superiority of offline supervised learning in terms of both optimality and training time. To overcome the main challenges, dataset and optimization, in the two approaches respectively, we complement them and propose the Pre-train and Fine-tune strategy as a unified training paradigm for optimal feedback control, which further improves the performance and robustness significantly. Our code is accessible at https://github.com/yzhao98/DeepOptimalControl.

Optimal Pose Design for Close-Proximity On-Orbit Inspection

Close-proximity on-orbit inspection is a critical ability to initiate on-orbit servicing operations, required technology for space exploration missions. It is challenging to solve an optimal inspection trajectory planning problem that provides a complete target observation due to major difficulties. First, the inspection requirements must be defined and imposed on the problem as a set of path constraints that result in a nondeterministic-polynomial-time-hard problem. Second, the optimization problem, including highly nonconvex constraints, is very difficult to solve directly using an optimal control solver. Additionally, it requires proper initialization of states and control variables, which is critical in such problems. To overcome these difficulties, this paper proposes a novel formulation and method of solution for a full six-degrees-of-freedom optimal inspection motion planning problem. Optimal inspection trajectories are designed for a rigid-body spacecraft, which performs close, continuous, and complete observation of rigid, nonrotating, and nonaccelerating known targets. The inspection and collision avoidance constraints are defined in explicit forms that rectify the nondifferentiability of the problem and satisfy the inspection requirements. A pseudospectral optimal control solver is implemented to numerically solve the trajectory optimization problem. The proposed methodology is applicable to any robotic inspection mission. Simulations are presented as a validation of the proposed methodology and the achieved optimality.

Fast and memory-efficient optimization for large-scale data-driven predictive control

Recently, data-enabled predictive control (DeePC) schemes based on Willems’ fundamental lemma have attracted considerable attention. At the core are computations using Hankel-like matrices and their connection to the concept of persistency of excitation. We propose an iterative solver for the underlying data-driven optimal control problems resulting from linear discrete-time systems. To this end, we apply factorizations based on the discrete Fourier transform of the Hankel-like matrices, which enable fast and memory-efficient computations. To take advantage of this factorization in an optimal control solver and to reduce the effect of inherent bad conditioning of the Hankel-like matrices, we propose an augmented Lagrangian BFGS-method. We illustrate the performance of our method by means of a numerical study.

A real-time capable method for planning minimum energy trajectories for one degree-of-freedom mechatronic systems

The synthesis of optimal motion profiles has shown to be a successful and virtually inexpensive solution for enhancing energy efficiency of mechatronic systems. A typical application is the design of point-to-point motion profiles for one-degree-of-freedom mechatronic systems. This paper proposes a new method for designing minimum energy trajectories for servo-actuated systems. The problem is solved by exploiting the knowledge of the structure of the optimal solution. That allows to solve the motion design problem by solving a set of nonlinear equations and, if needed, some basic optimization procedures, formulated after some suitable continuity conditions.The herein proposed method applies to systems with and without energy regeneration capability, for maximum adaptability to most industrial applications. The method also handles jerk, acceleration, and velocity constraints, which are typical requirements in many practical applications. The number of equations and thereby the computational time depends on the number of active constraints and on whether negative power is dissipated or regenerated. Overall, the method results to be suitable for Real-Time applications, also in the most challenging case in which all the constraints are active and the system cannot regenerate negative electric power. The accuracy and the effectiveness of the planning method is tested numerically, by comparing the solution to the one obtained by a general purpose optimal control solver and then also experimentally using a lab prototype.

GOPS: A general optimal control problem solver for autonomous driving and industrial control applications

Solving optimal control problems serves as the basic demand of industrial control tasks. Existing methods like model predictive control often suffer from heavy online computational burdens. Reinforcement learning has shown promise in computer and board games but has yet to be widely adopted in industrial applications due to a lack of accessible, high-accuracy solvers. Current Reinforcement learning (RL) solvers are often developed for academic research and require a significant amount of theoretical knowledge and programming skills. Besides, many of them only support Python-based environments and limit to model-free algorithms. To address this gap, this paper develops General Optimal control Problems Solver (GOPS), an easy-to-use RL solver package that aims to build real-time and high-performance controllers in industrial fields. GOPS is built with a highly modular structure that retains a flexible framework for secondary development. Considering the diversity of industrial control tasks, GOPS also includes a conversion tool that allows for the use of Matlab/Simulink to support environment construction, controller design, and performance validation. To handle large-scale problems, GOPS can automatically create various serial and parallel trainers by flexibly combining embedded buffers and samplers. It offers a variety of common approximate functions for policy and value functions, including polynomial, multilayer perceptron, convolutional neural network, etc. Additionally, constrained and robust algorithms for special industrial control systems with state constraints and model uncertainties are also integrated into GOPS. Several examples, including linear quadratic control, inverted double pendulum, vehicle tracking, humanoid robot, obstacle avoidance, and active suspension control, are tested to verify the performances of GOPS.

Adams-based hierarchical features fusion network for image dehazing

Recent developments in Convolutional Neural Networks (CNNs) have made them one of the most powerful image dehazing methods. In particular, the Residual Networks (ResNets), which can avoid the vanishing gradient problem effectively, are widely deployed. To understand the success of ResNets, recent mathematical analysis of ResNets reveals that a ResNet has a similar formulation as the Euler method in solving the Ordinary Differential Equations (ODE’s). Hence, image dehazing which can be formulated as an optimal control problem in dynamical systems can be solved by a single-step optimal control method, such as the Euler method. This optimal control viewpoint provides a new perspective to address the problem of image restoration. Motivated by the advantages of multi-step optimal control solvers in ODE’s, which include better stability and efficiency than single-step solvers, e.g. Euler, we propose the Adams-based Hierarchical Feature Fusion Network (AHFFN) for image dehazing with modules inspired by a multi-step optimal control method named the Adams–Bashforth method. Firstly, we extend a multi-step Adams–Bashforth method to the corresponding Adams block, which achieves a higher accuracy than that of single-step solvers because of its more effective use of intermediate results. Then, we stack multiple Adams blocks to mimic the discrete approximation process of an optimal control in a dynamical system. To improve the results, the hierarchical features from stacked Adams blocks are fully used by combining Hierarchical Feature Fusion (HFF) and Lightweight Spatial Attention (LSA) with Adams blocks to form a new Adams module. Finally, we not only use HFF and LSA to fuse features, but also highlight important spatial information in each Adams module for estimating the clear image. The experimental results using synthetic and real images demonstrate that the proposed AHFFN obtains better accuracy and visual results than that of state-of-the-art methods.

Low-Thrust, Multiple-Near-Earth-Asteroid Mission Design with Sample Return to Earth Using Machine Learning

Sample return missions to near-Earth asteroids (NEAs) are invaluable for the scientific community to learn more about the initial stages of the solar system formation and life evolution. Low-thrust propulsion technology enables missions with multiple-asteroid rendezvouses to collect samples and eventually return to Earth, thanks to its high specific impulse. To identify the best asteroid sequences with return to Earth, this work proposes to employ machine learning techniques and, specifically, artificial neural networks (ANNs), to quickly estimate the cost of each transfer between asteroids. The ANN is integrated within a sequence search algorithm based on a tree search, which identifies the asteroid sequences and selects the best ones in terms of propellant mass required and interest value. This algorithm can design NEA sequences so that specific asteroids of interest, for which a sample return would be more valuable, can be targeted. A pseudospectral optimal control solver is then used to find the optimal trajectory and control history. The performance of the proposed methodology is assessed by analyzing three distinctive NEA sequences ending with return to Earth and rendezvous. Near-term low-thrust propulsion enables to rendezvous five asteroids, and return samples to Earth in about 10 years from launch. It is demonstrated that visiting more scientifically interesting asteroids increases the appeal of the sequence at the cost of more propellant mass required.

Optimal Control for Kinematic Bicycle Model With Continuous-Time Safety Guarantees: A Sequential Second-Order Cone Programming Approach

The optimal control problem for the kinematic bicycle model is considered where the trajectories are required to satisfy the safety constraints in the continuous-time sense. Based on the differential flatness property of the model, necessary and sufficient conditions in the flat space are provided to guarantee safety in the state space. The optimal control problem is relaxed to the problem of solving three second-order cone programs (SOCPs) sequentially, which find the safe path, the trajectory duration, and the speed profile, respectively. Solutions of the three SOCPs together provide a sub-optimal solution to the original optimal control problem. Simulation examples and comparisons with state-of-the-art optimal control solvers are presented to demonstrate the effectiveness of the proposed approach.

Stability-constrained mobile manipulation planning on rough terrain

AbstractThis paper presents a hierarchical framework that allows online point-to-point dynamic-stability-constrained optimal trajectory planning of a mobile manipulator robot working on rough terrain. First, the kinematics model of a mobile manipulator robot and the zero moment point stability measure are presented as theoretical background. Then, a sampling-based quasi-static planning algorithm modified for stability guarantee and traction optimization in continuous dynamic motion is presented along with a mathematical proof. The robot’s quasi-static path is then used as an initial guess to warm start a nonlinear optimal control solver which may otherwise have difficulties finding a solution to the stability-constrained formulation efficiently. The performance and computational efficiency of the framework are demonstrated through an application to a simulated timber harvesting mobile manipulator machine working on varying terrain. The results demonstrate feasibility of online trajectory planning on varying terrain while satisfying the dynamic stability constraint. Qualitative and quantitative comparisons with existing methods are also presented.

Time optimal control of a non-linear surface vehicle subject to disturbances

The problem of an autonomous agent moving on a planar surface, such as an aerial drone or a small naval vessel can be treated as navigation between a series of points. While nominally the movement between each pair of points can be treated as a 1D projection of the movement on the vector connecting the two points, in the presence of disturbances, the full problem on the plane must be considered. The time optimal solution is now dependent on the value and direction of the disturbance which in this paper is assumed to be a constant inertial velocity of the medium (wind or current, respectively). We address the minimum time problem of movement on a 2D plane with quadratic drag, under norm state (inertial vessel velocity), and norm control (acceleration) constraints. The structure and properties of the optimal solution are found and analyzed, utilizing the Pontryagin Maximum Principle (PMP) with control and state constraints. Simulations supporting the results are provided and compared with those of the open-source academic optimal control solver Falcon.m.

A Nonlinear Safety Equilibrium Spacing-Based Model Predictive Control for Virtually Coupled Train Set Over Gradient Terrains

The increasing demand for capacity in railway transportation has spawned the concept of virtual coupling (VC), which can further shorten the intertrain distance by applying communication technology. In this article, a centralized model predictive control (MPC) is developed for virtually coupled train set (VCTS) with a nonlinear safety equilibrium spacing policy over gradient rolling terrains. The tracking target of the controller in cruise phase is given based on the braking distance difference and speed difference of trains. A more precise braking distance model of trains explicitly considering the gradient is introduced for safety, based on which the state space applicable to the controller for VCTS is derived. To deal with the nonlinearities of the controller, a new optimization solving algorithm based on the continuation method and generalized minimum residual (GMRES) method is presented in this article. Finally, controller performances and empirical string stability of VCTS are verified by numerical simulations. It is proven that the proposed controller has a good effect for the safety and smoothness of the train operation on the railway line over gradient terrain through the test. The actual intertrain distance under the proposed controller is about 1 m closer to the equilibrium state than that under the same controller without considering the gradient at low speed (about 10 m/s) on the 0.3% downhill line, which verifies the safety of the controller. Besides, the influence of the parameters of the controller and optimization method on controller efficiency is analyzed, and for the latter, the simulation data show that the applied algorithm for the optimal control solver can save the calculation time more than seven times than the traditional algorithm.

Traffic-Prediction-Based Optimal Control of Electric and Autonomous Buses

This work considers electric and autonomous buses which have to follow a given route, including fixed stops, in extra-urban roads with a given timetable. A charging infrastructure is present in each stop, allowing to charge the bus batteries. An optimal control scheme is proposed in this paper in order to regulate the optimal speed of buses along the route and the stopping/charging times at stops. The proposed control scheme, acting in real time according to a receding-horizon logic, consists of two modules: a traffic prediction model and an optimal control problem solver. The traffic model measures the traffic state in real time, provides the traffic state prediction in the considered road stretch and, in particular, communicates the predicted average speed in each road section to the second module. This latter computes the optimal behavior of buses by optimizing their expected final energy level, by maximizing their compliance with the timetable and by reducing oscillations in the speed profile. Simulation results based on a real case study show the effectiveness of the proposed control scheme.

Empowering Optimal Control with Machine Learning: A Perspective from Model Predictive Control

Solving complex optimal control problems have confronted computational challenges for a long time. Recent advances in machine learning have provided us with new opportunities to address these challenges. This paper takes model predictive control, a popular optimal control method, as the primary example to survey recent progress that leverages machine learning techniques to empower optimal control solvers. We also discuss some of the main challenges encountered when applying machine learning to develop more robust optimal control algorithms.

Direct optimal control for time-delay systems via a lifted multiple shooting algorithm

<p style='text-indent:20px;'>Aiming at efficient solution of optimal control problems for continuous nonlinear time-delay systems, a multiple shooting algorithm with a lifted continuous Runge-Kutta integrator is proposed. This integrator is in implicit form to remove the restriction of smaller integration step sizes compared with delays. A tangential predictor is applied in the integrator such that Newton iterations required can be reduced considerably. If one Newton iteration is applied, the algorithm has the same structure as direct collocation algorithms whereas derives a condensed nonlinear programming problem. Then, the solution of variational sensitivity equation is decoupled from forward simulation by utilizing the implicit function theorem. Under certain conditions, this function evaluation and derivative computation procedure is proved to be convergent with a global order. Complexity analysis shows that the computational cost can be largely reduced by this lifted multiple shooting algorithm. Then, parallelizable optimal control solver can be constructed by embedding this algorithm in a general-purpose nonlinear programming solver. Simulations on a numerical example demonstrate that the computational speed of multi-threading implementation of this algorithm is increased by <inline-formula><tex-math id="M1">\begin{document}$ 36\% $\end{document}</tex-math></inline-formula> compared with non-lifted one, and increased by a factor of <inline-formula><tex-math id="M2">\begin{document}$ 6.64 $\end{document}</tex-math></inline-formula> compared with traditional sequential algorithm; meanwhile, the accuracy loss is negligible.</p>

Real-Time Trajectory Planning for Automated Vehicle Safety and Performance in Dynamic Environments

Abstract Safe trajectory planning for high-performance automated vehicles in an environment with both static and moving obstacles is a challenging problem. Part of the challenge is developing a formulation that can be solved in real-time while including the following set of specifications: minimum time-to-goal, a dynamic vehicle model, minimum control effort, both static and moving obstacle avoidance, simultaneous optimization of speed and steering, and a short execution horizon. This paper presents a nonlinear model predictive control-based trajectory planning formulation, tailored for a large, high-speed unmanned ground vehicle, that includes the above set of specifications. The ability to solve this formulation in real-time is evaluated using NLOptControl, an open-source, direct-collocation-based optimal control problem solver in conjunction with the KNITRO nonlinear programming problem solver. The formulation is tested with various sets of specifications. A parametric study relating execution horizon and obstacle speed indicates that the moving obstacle avoidance specification is not needed for safety when the planner has a small execution horizon and the obstacles are moving slowly. However, a moving obstacle avoidance specification is needed when the obstacles are moving faster, and this specification improves the overall safety without, in most cases, increasing the solve-times. The results indicate that (i) safe trajectory planners for high-performance automated vehicles should include the entire set of specifications mentioned above, unless a static or low-speed environment permits a less comprehensive planner and (ii) the resulting formulation can be solved in real-time.

Chance-constrained path planning in narrow spaces for a dubins vehicle

The problem of optimal path planning through narrow spaces in an unstructured environment is considered. The optimal path planning problem for a Dubins agent is formulated as a chance-constrained optimal control problem (CCOCP), wherein the uncertainty in obstacle boundaries is modelled using standard probability distributions. The chance constraints are transformed to deterministic equivalents using the inverse cumulative distribution function and subsequently incorporated into a deterministic optimal control problem. Due to multiple convex sub-regions introduced by the obstacles, the initial guess provided to optimal control solver is crucial for computation time and optimality of the solution. A constrained Delaunay triangulation mesh based approach is developed that ensures the initial guess to lie in the optimal sub-convex region. Finally, off-the-shelf software is used to transcribe the optimal control problem to a nonlinear program (NLP) using Gaussian quadrature orthogonal collocation and solved to obtain an optimal path that conforms to system dynamics. By varying the upper bound on probability of obstacle collision, a family of solutions is generated, parameterized by the risk associated with each solution. This approach enables discovery of special “keyhole trajectories” that can provide significant cost savings in a tightly-spaced obstacle field. Merits of this approach are illustrated by comparing it with the traditional bounded uncertainty approach.

Memory Clustering Using Persistent Homology for Multimodality- and Discontinuity-Sensitive Learning of Optimal Control Warm-Starts

Shooting methods are an efficient approach to solving nonlinear optimal control problems. As they use local optimization, they exhibit favorable convergence when initialized with a good warm-start but may not converge at all if provided with a poor initial guess. Recent work has focused on providing an initial guess from a learned model trained on samples generated during an offline exploration of the problem space. However, in practice, the solutions contain discontinuities introduced by system dynamics or the environment. Additionally, in many cases, multiple equally suitable, i.e., multimodal, solutions exist to solve a problem. Classic learning approaches smooth across the boundary of these discontinuities and thus generalize poorly. In this work, we apply tools from algebraic topology to extract information on the underlying structure of the solution space. In particular, we introduce a method based on persistent homology to automatically cluster the dataset of precomputed solutions to obtain different candidate initial guesses. We then train a mixture-of-experts within each cluster to predict state and control trajectories to warm-start the optimal control solver and provide a comparison with modality-agnostic learning. We demonstrate our method on a cartpole toy problem and a quadrotor avoiding obstacles, and show that clustering samples based on inherent structure improves the warm-start quality.

Optimization-based subdivision algorithm for reachable sets

<p style='text-indent:20px;'>Reachable sets for nonlinear control systems can be computed via the use of solvers for optimal control problems. The paper presents a new improved variant which applies adaptive concepts similar to the framework of known subdivision techniques by Dellnitz/Hohmann. Using set properties of the nearest point projection, the convergence and rigorousness of the algorithm can be proved without the assumption of diffeomorphism on a nonlinear mapping. The adaptive method is demonstrated by two nonlinear academic examples and for a more complex robot model with box constraints for four states, two controls and five boundary conditions. In these examples adaptive and non-adaptive techniques as well as various discretization methods and optimization solvers are compared. <p style='text-indent:20px;'>The method also offers interesting features, like zooming into details of the reachable set, self-determination of the needed bounding box, easy parallelization and the use of different grid geometries. With the calculation of a 3d funnel in one of the examples, it is shown that the algorithm can also be used to approximate higher dimensional reachable sets and the resulting box collection may serve as a starting point for more sophisticated visualizations or algorithms.