Indirect Optimization Method Research Articles

Robust solar sail trajectories using proximal policy optimization

Reinforcement learning is used to design minimum-time trajectories of solar sails subject to the typical sources of uncertainty associated with such a propulsion system, i.e., inaccurate knowledge of the sail’s optical properties and the presence of wrinkles on the sail membrane. A proximal policy optimization (PPO) algorithm is used to train the agent and derive the control policy that associates the optimal sail attitude with each dynamic state. First, the agent is trained assuming deterministic unperturbed dynamics, and the results are compared with optimal solutions found by an indirect optimization method, thus demonstrating the effectiveness of this approach. Next, two stochastic scenarios are analyzed. In the first, the optical coefficients of the sail are assumed to be random variables with Gaussian distribution, which leads to random variations in the sail characteristic acceleration. In the second scenario, wrinkles on the sail membrane are taken into account, resulting in a misalignment of the thrust vector with respect to a perfectly smooth surface. Both phenomena are modeled based on experimental measurements available in the literature in order to perform realistic analyses. In the stochastic scenarios, Monte Carlo simulations are performed using the trained policies, demonstrating that the reinforcement learning approach is capable of finding near time-optimal solutions, while also being robust to the sources of uncertainty considered.

Neural Network-Based Optimization of LEO Transfers

This study investigates the application of neural networks to the evaluation of minimum-time low-thrust transfers in low Earth orbit. The findings demonstrate the effectiveness of utilizing costates to regularize the training loss, significantly enhancing the accuracy of the predictions of the neural networks, even when working with limited datasets. Remarkably precise estimates of transfer times are achieved by training the regularized networks on datasets comprising one million samples. The incorporation of a warm-started guess strategy, involving simpler neural networks to provide transfer time and costates predictions for new transfers, accelerates the data collection process, making this approach highly practical for real-world applications. Overall, the methodology employed in this research study holds significant promise for low-thrust space missions, particularly when the evaluation of multiple minimum-time transfers is necessary in mission planning. In fact, the trained neural networks significantly speed up convergence when solving optimal control problems with indirect optimization methods. Furthermore, the remarkable accuracy in estimating both minimum transfer times and costates provides the flexibility of relying entirely on neural networks for determining minimum time.

MODERNIZING CLASSICAL OPTIMAL CONTROL: HARNESSING DIRECT AND INDIRECT OPTIMIZATION

An introduction to optimal control, a fundamental concept in engineering and science disciplines, is a process of finding ways of controlling dynamic systems in such a way that they achieve certain goals while being subjected to given state limitations. The conventional approaches developed in the past have been integral to solving optimal control problems, including the maximum principle belonging to Pontryagin and the dynamic programming method. While relatively straightforward, these methods are not always amenable to higher-dimensional scenarios, interacting forces, or other non-trivial constraints. This paper presents a new methodology to extend classical optimal control, considering both direct and indirect optimization techniques. The direct methods, Euler and Runge-Kutta, Trapezoidal, and Hermite-Simpson, do not require the explicit derivation of the analytical control laws to execute control trajectories. Semi-analytical techniques, such as the shooting method, are based the control laws on proposed adjoint differential equations. This paper presents the basic outline of the classical optimal control problem and discusses the direct and indirect optimization methods. It is illustrated by referencing various examples, like the fixed-rate royalty payment approach. After outlining each framework, we identify the positive and negative aspects of the approach in questions of consistency and performance. Finally, the problems on the synergy of direct and indirect methods are considered further, and areas of further development of the presented methods and their integration with the more sophisticated tools, such as machine learning are identified. It can be concluded that the approach of using both direct and indirect optimization methods presents great potential in modernizing the classical optimal control, which challenges the conventional techniques and indicates potential for further development of the control system optimization.

Low-Thrust Trajectory Optimization with Averaged Dynamics Using Analytical Switching Detection

This paper presents an efficient indirect optimization method for the time- and fuel-optimal low-thrust many-revolution transfers with averaged dynamics. Compared with the previous indirect method, the proposed method involves a new set of variables and several near-optimal control laws to formulate the two-point boundary value problem based on three practical assumptions. Then, the costate transformations between two reference frames are first derived with respect to the the corresponding Eulerian angles, such that the switching function can be expressed with a six-degree polynomial function and the number of parameters used to determine the energy-optimal solutions is reduced to 6. Finally, a neural network is trained to generated the initial guesses, and the solution algorithms for solving the time- and fuel-optimal problem are formulated. Numerical simulations demonstrate that the convergence ratio of initial guesses is higher than 98%, the proposed assumptions are reasonable for guaranteeing the near optimality, and the presented method can obtain convergent solutions within tenths of a second for the Earth- and moon-centered external/interior transfers considering the shadow J2 and third-body perturbations.

Duty-cycle-aware low-thrust trajectory optimization using embedded homotopy

In space mission design, an important operational constraint is planned periodic coast arcs during which various mission-critical tasks such as navigation or ground communications are performed. These constraints, typically implemented as thruster duty cycles, can be characterized as discrete coast arcs enforced periodically during intervals of deterministic thrusting. A typical approach to incorporate duty cycles is by enforcing maximum thrust levels that are less than a nominal value. This work presents an embedded homotopy indirect optimization method for a smooth enforcement of discrete duty-cycle constraints for low-thrust trajectory optimization. The complexity of enforcing such discrete constraints is two-fold: (1) a precise event-detection strategy is needed to determine the beginning of deterministic thrust arcs, which must be successively updated along the trajectory and (2) a smooth modeling of the discrete coast events occurring on the order of 10s or even 100s depending on the duration of deterministic thrust arcs. Duty cycle constraints are enforced at the level of the Hamiltonian using a Composite Smooth Control (CSC) method, producing high-resolution optimal control solutions conscious of planned duty cycle coast arcs. Numerical solutions are presented for the heliocentric transfer phases of three space missions. The results are then validated against the Evolutionary Mission Trajectory Generator (EMTG), one of NASA’s global trajectory optimization tools used for interplanetary mission design and capable of the explicit enforcement of periodic duty cycle coasting. The proposed indirect method is suitable for obtaining high-resolution trajectory solutions with realistically enforced thruster duty cycle, which could then be used as high-quality initial guesses in flight-fidelity mission planning tools. The proposed smooth modeling of discrete coast arcs can also be customized to enforce periodic and non-periodic thrust shutdown constraints.

Emission-Driven Hybrid Rocket Engine Optimization for Small Launchers

Hybrid rocket engines are a green alternative to solid rocket motors and may represent a low-cost alternative to kerosene fueled rockets, while granting performance and control features similar to that of typical storable liquid rocket engines. In this work, the design of a three-stage hybrid launcher is optimized by means of a coupled procedure: an evolutionary algorithm optimizes the engine design, whereas an indirect optimization method optimizes the corresponding ascent trajectory. The trajectory integration also provides the vertical emission profiles required for the evaluation of the environmental impact of the launch. The propellants are a paraffin-based wax and liquid oxygen. The vehicle is launched from the ground and uses an electric turbo pump feed system. The initial mass is given (5000 kg) and the insertion of the payload into a 600-km circular, and polar orbit is considered as a reference mission. Clusters of similar hybrid rocket engines, with only few differences, are employed in all stages to reduce the development and operational costs of the launcher. Optimization is carried out with the aim of maximizing the payload mass and then minimizing the overall environmental impact of the launch. The results show that satisfactory performance is achievable also considering rocket polluting emissions: the carbon footprint of the launch can be reduced by one fourth at the cost of a 5-kg payload mass reduction.

Minimum-fuel geostationary east-west station-keeping using a three-phase deep neural network

With the development of Electric Propulsion (EP) systems in the geostationary region, indirect optimization method shows a great potential in solving the station-keeping control law. Three main challenges, including the strong nonlinearity of the equations of motion for both states and costate, the consideration of the free-transfer-time constraint, and the Earth-shadow prediction, are hindering the autonomous control onboard. In this study, a three-phase Deep Neural Network (DNN) with three modules is proposed to achieve autonomous station-keeping by providing the optimal transfer time, the prediction of the Earth's shadow, and the initial costate guess for the East-West Station Keeping (EWSK) mission. Compared with the traditional method, the efficiency of the proposed solver is significantly increased since the minimum-fuel problem is directly solved without calculating the minimum-time solution. Numerical assessment results can indicate that the DNN could provide the parameters accurately with high scalability. Particularly, an expected success rate can be obtained to validate the autonomous EWSK strategy.

Topologies comparison and rotor optimization of the hybrid axial transverse flux permanent magnet motor

With the advantages of high power density, the hybrid axial transverse flux permanent magnet motor (HATF-PMM) is investigated in this paper. Eight topologies of the novel HATF-PMM are proposed, analysed and compared. The applicable scopes of different topologies are summarized and the optimal topology is selected. Based on the characteristics of hybrid flux motor, an indirect optimization method through magnetic circuit decoupling and recombination is proposed. The rotor parameter optimization is performed by the proposed indirect optimization method. In the end, the HATF-PMM scheme with high power density is obtained, and the optimization law of HATF-PMM rotor parameters is revealed through the optimization process, which proves that the HATF-PMM has great potential in the application of more-electric aircraft.

Time-energy optimal landing on planetary bodies via theory of functional connections

In this paper, we propose a unified approach to solve the time-energy optimal landing problem on planetary bodies (e.g. planets, moons, and asteroids). In particular, the indirect optimization method, based on the derivation of the first order necessary conditions from the Hamiltonian, is exploited and the Two-Point Boundary Value Problem arising from the application of the Pontryagin Minimum Principle is solved using the Theory of Functional Connections. The optimal landing trajectories are accurately computed with a computational time on the order of 10–100 ms, using a MATLAB implementation. The speed and accuracy of the proposed method makes it suitable for real time applications. The algorithm is applied and validated for the landing on large (Mars and Moon) and small (asteroids Gaspra and Bennu) planetary bodies.

Optimal Escape from Sun-Earth and Earth-Moon L2 with Electric Propulsion

Optimal low-thrust trajectories for the direct escape from the Earth’s sphere of influence, starting from Sun-Earth or Earth-Moon L2, are analyzed with an indirect optimization method. The dynamic model considers four-body gravitation and JPL ephemeris; solar radiation pressure is also considered. Specific techniques and improvements to the method are introduced to tackle the highly chaotic and nonlinear dynamics of motion close to Lagrangian points, which challenges the remarkable precision of the indirect method. The results show that escape trajectories have optimal performance when the solar perturbation acts favorably in both thrust and coast phases. The effects of the solar and Moon perturbations are more evident in the Earth-Moon L2 escapes compared with those from the Sun-Earth L2. EML2 escapes have single- or two-burn solutions depending on the trajectory deflection, which is needed to have a favorable solar perturbation. The SEL2 escapes, on the contrary, mainly have a single initial burn and a long coast arc, but need an additional final thrust arc if the required C3 is high. As applications of such Lagrangian Point trajectories, results include considerations about escape maneuvers from different SEL2 high-fidelity Lyapunov orbits and escape for interplanetary trajectories towards near-earth asteroids.

Approximate optimal LEO transfers with J2 perturbation and dragsail

An indirect optimization method is applied to orbit transfers in LEO, considering almost circular orbits and the influence of J2 perturbation. An approach based on Edelbaum’s approximation is employed to solve transfers with change of semimajor axis, inclination and right ascension of ascending node. The spacecraft employs electric propulsion either alone or in combination with a drag sail that can be deployed and retracted. The proposed formulation is simple and effective and is capable of treating both minimum-time (thruster is always on) and minimum-propellant (coasting arcs are introduced) problems, while also dealing with the presence of altitude constraints. Convergence to the optimal solution is fast and straightforward, making the proposed approach suitable for the preliminary evaluation of large sets of available transfers (e.g., multiple debris removal).

Learning-based optimization of acquisition schedule for magnetization transfer contrast MR fingerprinting.

Magnetization transfer contrast MR fingerprinting (MTC-MRF) is a novel quantitative imaging method that simultaneously quantifies free bulk water and semisolid macromolecule parameters using pseudo-randomized scan parameters. To improve acquisition efficiency and reconstruction accuracy, the optimization of MRF sequence design has been of recent interest in the MRF field, but has been challenging due to the large number of degrees of freedom to be optimized in the sequence. Herein, we propose a framework for learning-based optimization of the acquisition schedule (LOAS), which optimizes RF saturation-encoded MRF acquisitions with a minimal number of scan parameters for tissue parameter determination. In a supervised learning framework, scan parameters were subsequently updated to minimize a predefined loss function that can directly represent tissue quantification errors. We evaluated the performance of the proposed approach with a numerical phantom and in in vivo experiments. For validation, MRF images were synthesized using the tissue parameters estimated from a fully connected neural network framework and compared with references. Our results showed that LOAS outperformed existing indirect optimization methods with regard to quantification accuracy and acquisition efficiency. The proposed LOAS method could be a powerful optimization tool in the design of MRF pulse sequences.

Optimization for Far-distance Cooperative Rendezvous with Multiple Direction-fixed Thrusts

The continuous-thrust far-distance cooperative rendezvous problem between two spacecraft is investigated. The indirect optimization method, based on Pontryagin’s maximum principle (PMP), is applied to optimize fuel consumption. To overcome the difficulty in nonsmooth integration caused by the bang-bang control, the homotopy method is adopted to solve the fuel-optimal problem from a related energy-optimal problem. The quantum-behaved particle swarm optimization (QPSO) algorithm is used to obtain the energy-optimal solutions. The energy-optimal solutions are used as the initial values for the homotopic procedure to obtain the fuel-optimal solutions and optimal bang-bang control law. A hybrid algorithm combined homotopy method and sequential quadratic programming (SQP) algorithm is proposed. This hybrid algorithm can effectively obtain feasible optimal solutions even though the indirect optimization method exhibits a narrow convergence domain. Simulations of high-thrust and low-thrust rendezvous problems are provided and the proposed hybrid algorithm is verified. Moreover, the necessity of radial thrust is investigated.

Fast and accurate estimation of fuel-optimal trajectories to Near-Earth Asteroids

This paper proposes an improved method for the preliminary evaluation of minimum-propellant trajectories to Near-Earth Asteroids (NEAs). The method applies to missions from Earth to asteroids with small eccentricity and inclination. A planar and a plane-change problem can be distinguished. In the planar problem, the solution assumes that multiple burn arcs are performed in correspondence of the apsides of the target asteroid in order to change the initial spacecraft orbit (i.e., Earth’s orbit) into the target one. The number of arcs is established once the time of flight is given (1 burn at each apsis per revolution, 1 revolution per year can be assumed). The length and propellant consumption of each arc to attain the required changes of semi-major axis and eccentricity are computed by a procedure based on Edelbaum’s approximation, which is well-suited to the problem at hand, as eccentricity changes are expected to be small for feasible missions. No numerical integration is required, but only the numerical solution of a three-unknown algebraic system is needed, making the procedure extremely fast. Plane change is taken into account assuming a constant out-of-plane thrust angle during each burn. A previous simple formulation used an averaged thrust effect over one revolution and neglected the fact that plane changes are more effective at the nodes. Several improvements are here introduced, which greatly increase the method accuracy. The influences of the eccentricity change, the angle between the asteroid line of nodes and line of apsides, and the expected length of the arc are considered: In fact, when the eccentricity is small, the thrust arc can be performed at the nodes where the inclination is efficiently changed, with little penalty in the planar maneuver. An efficient plane change is also performed when the angle between the asteroid line of nodes and line of apsides is small and/or the length of the arc is large, because, in this case, the node is comprised in the apsidal burn. A simple corrective formula accounts for this effect. The new method shows remarkable accuracy. The results comparison with solutions obtained with an indirect optimization method for a set of more than 60 NEAs shows a 0.95 correlation coefficient in the propellant masses. The estimation error is below 10% for 75% of the targets, below 15% for 95% of the targets, and always below 20%.

Low-Thrust Gravity-Assist Trajectory Design Using Optimal Multimode Propulsion Models

The complexities in using indirect optimization methods get compounded for practical co-optimization problems in the presence of continuous and discrete design variables. In this paper, realistic multimode electric propulsion systems are incorporated within the formulation of gravity-assist, low-thrust trajectory design problems. Electric thrusters operate over a large set of discrete operation modes, and each mode is characterized by specific values for power, thrust, specific impulse, and mass flow rate. Unlike the traditional methods that approximate the aggregate behavior of electric thrusters’ performance (using polynomials fits), a novel construct is proposed to incorporate optimal discrete operating modes within indirect optimization. The capability of the tool and the efficacy of the proposed methodology are demonstrated by solving a multiyear, fuel-optimal trajectory problem from Earth to asteroid Psyche via a Mars gravity-assist maneuver and using a Hall-effect SPT-140 thruster with 21 operating modes. Comparisons with the traditional polynomial-based thruster modeling are presented.

Trajectory control methods for observation in two-position angular airborne radio monitoring systems Part 1. Orthogonal methods

Formulation of the problem. Currently, much attention is paid to the practical application and further improvement of two-position goniometric systems (TPGS) airborne radio monitoring, the advantage of which over single-position systems is the possibility of almost instantaneous determination of the location of the radio emission source (RES). A feature of the operation of the TPGS is the dependence of the errors in the positioning of the RES on the «geometry» of the system, i.e. on the size of the base and the position of the RES relative to the base. That is why the problem of optimizing the mutual arrangement of the receiving positions and RES is very urgent. To minimize positioning errors, two groups of trajectory observation control (TOC) methods have been developed, which is due to the variety of conditions for the practical use of aviation goniometric two-position systems and different degrees of their effectiveness. The first group of methods, called orthogonal, provides control of the receiving positions placed on the aircraft, so that the lines of sight of the RES relative to them intersect at right angles. The formation of a right-angled triangulation triangle, at the vertices of which the aircraft and RES are located, can be ensured by appropriate movement of the information support aircraft, or by simultaneously pointing both aircraft. In the first case, it is envisaged to output the information aircraft to an arbitrary point of a straight line, orthogonal to the line of sight «leading aircraft-RES», during the flight along which the best conditions for observation are created, or to a given point on this line. The second group of TOC methods ensures the minimization of errors in the positioning of radioactive sources at each current moment of time. When developing the corresponding trajectory control algorithms that implement the indicated trajectory control methods, direct and indirect optimization methods were used. To date, various TOC methods have been developed, but there is no systematic presentation of them. Purpose. To give a systematized idea of the developed methods of trajectory control of observation in two-position goniometric systems for radio monitoring of the surrounding space, as well as ways of solving the problem of ensuring the best observation conditions and increasing the accuracy of estimating the phase coordinates of the RES in these systems. Results. In the first part of this article, orthogonal methods and algorithms of TOC are considered, the implementation of which contributes to a significant increase in the accuracy of estimating the coordinates of the RES in goniometric two-position airborne radio monitoring systems. Information on the results of experimental studies confirming the effectiveness of the developed TOC methods is also given. Practical significance. The areas of practical application of various TOC methods in solving radio monitoring problems are shown, information about which can be used when choosing rational trajectory control methods for specific conditions of practical application, as well as when developing new TOC methods.

A composite framework for co-optimization of spacecraft trajectory and propulsion system

Indirect optimization methods convert optimal control problems (OCPs) into two- or multi-point boundary-value problems. A highly desirable feature of indirect methods, specifically for space applications, is that high-resolution trajectories can be generated, which satisfy the first-order necessary conditions of optimality. A recently developed Composite Smoothing Control (CSC) framework is utilized to formulate and solve the problem of simultaneous trajectory optimization and propulsion sub-system design of spacecraft. A reasonable parameterized breakdown of the spacecraft mass is adopted, which captures the impact of power produced by the solar arrays and its contribution to the total spacecraft mass. Thus, the implicit trade-offs can be considered in the indirect optimization approach. The function space co-optimization problem of spacecraft power subsystem parameters along with the main trajectory is solved with the objective to maximize the payload delivered. The proposed framework amounts to an invariant embedding that reduces the original, difficult-to-solve, multi-point boundary-value problem into a two-point boundary-value problem with continuous, differentiable control inputs. Utility of the proposed construct is demonstrated through a low-thrust, multi-revolution, multi-year rendezvous maneuver to asteroid Dionysus with a variable-specific-impulse, variable-thrust modeled engine. This is the first time that indirect optimization methods have tackled such a complex co-optimization problem using the CSC framework.

Indirect Optimization of Underactuated Spacecraft Formation Reconfiguration in Elliptic Orbits

This paper proposes analytical solutions to optimal underactuated spacecraft formation reconfiguration in elliptic orbits, without either the radial or in-track thrust. For either underactuated case, the dynamical model of underactuated formation reconfiguration was developed, based on which the system controllability and reconfiguration feasibility then were analyzed. By using the indirect optimization method, a detailed optimization procedure was designed, and the optimal trajectory of formation reconfiguration was derived analytically for the underactuated cases. A fully actuated optimal reconfiguration scheme was introduced to make comparisons. Numerical simulations indicated that the proposed optimal underactuated reconfiguration schemes can manage formation reconfiguration with a control cost similar to that of the fully actuated scheme, even in the absence of radial or in-track thrust. Numerical comparisons with existing nonoptimal reconfiguration schemes further verified that the proposed optimal schemes can perform underactuated formation reconfiguration in an energy-optimal way.

Uniform Trigonometrization Method for Optimal Control Problems with Control and State Constraints

Application of indirect optimization methods to solve constrained optimal control problems is not a straightforward task. This study presents a new construct, the Uniform Trigonometrization Method (UTM), that enables handling problems with control bounds and state path constraints within the indirect formalism. First, three different classes of problems are discussed with 1) bang–bang and singular control arcs, where the control appears in a linear form; 2) control constraints in which the control appears in a nonlinear form; and 3) state path constraints. Accurate solutions are achieved in all classes of problems with a simpler formulation compared to the conventional indirect approach. To demonstrate the potential of the UTM, a complex aerospace problem with bounded control (angle of attack) and state path (maximum heat rate) constraints is also solved. The results demonstrate the utility of the UTM for making challenging constrained optimal control problems amenable to numerical treatment with minimal modifications to the structure of the problem.

Optimal two-impulse space interception with multiple constraints

We consider optimal two-impulse space interception problems with multiple constraints. The multiple constraints are imposed on the terminal position of a space interceptor, impulse and impact instants, and the component-wise magnitudes of velocity impulses. These optimization problems are formulated as multi-point boundary value problems and solved by the calculus of variations. Slackness variable methods are used to convert all inequality constraints into equality constraints so that the Lagrange multiplier method can be used. A new dynamic slackness variable method is presented. As a result, an indirect optimization method is developed. Subsequently, our method is used to solve the two-impulse space interception problems of free-flight ballistic missiles. A number of conclusions for local optimal solutions have been drawn based on highly accurate numerical solutions. Specifically, by numerical examples, we show that when time and velocity impulse constraints are imposed, optimal two-impulse solutions may occur; if two-impulse instants are free, then a two-impulse space interception problem with velocity impulse constraints may degenerate to a one-impulse case.