Earth-Mars Mission Research Articles

Densely rewarded reinforcement learning for robust low-thrust trajectory optimization

To overcome the time-consuming training caused by the sparse reward function in reinforcement learning, an efficient dense reward framework for robust low-thrust trajectory optimization is proposed. The dense reward functions are designed separately for the deterministic and considered uncertain scenarios, including state uncertainties, observation uncertainties and execution uncertainties. For the uncertainties, the dense reward function is designed to diminish the deviation with respect to the isochronous nominal state along the corresponding deterministic optimal trajectory at each step, rendering the reward function no longer sparse and suitable for complex multirevolution problems. In addition, a multistage reward function of the terminal constraints for the rendezvous missions is designed by incorporating some exponential acceleration terms, enabling significant improvement in training efficiency as the terminal errors become low. In addition, a dense reward function for the deterministic scenario is also proposed via the introduction of empirical forbidden zones and an exponential term. The effectiveness and efficiency of the proposed method is demonstrated in a simple Earth-Mars mission and a complex Earth-Venus multirevolution mission. The promising results verify the significant effect of the proposed method in speeding up the process of training an initial incapable agent to an ‘expert’ while guaranteeing or even improving the performance.

Reinforcement Learning for Robust Trajectory Design of Interplanetary Missions

This paper investigates the use of reinforcement learning for the robust design of low-thrust interplanetary trajectories in presence of severe uncertainties and disturbances, alternately modeled as Gaussian additive process noise, observation noise, and random errors in the actuation of the thrust control, including the occurrence of a missed thrust event. The stochastic optimal control problem is recast as a time-discrete Markov decision process to comply with the standard formulation of reinforcement learning. An open-source implementation of the state-of-the-art algorithm Proximal Policy Optimization is adopted to carry out the training process of a deep neural network, used to map the spacecraft (observed) states to the optimal control policy. The resulting guidance and control network provides both a robust nominal trajectory and the associated closed-loop guidance law. Numerical results are presented for a typical Earth–Mars mission. To validate the proposed approach, the solution found in a (deterministic) unperturbed scenario is first compared with the optimal one provided by an indirect technique. The robustness and optimality of the obtained closed-loop guidance laws is then assessed by means of Monte Carlo campaigns performed in the considered uncertain scenarios.

Optimal low-thrust hyperbolic rendezvous for Earth-Mars missions

Earth-Mars cycling spacecraft have been proposed as a valuable option for the future exploration of Mars. Cycler mission architectures consider the use of a large space vehicle that cycles continuously between Earth and Mars, describing a near-ballistic path that includes flybys at the two planets. While this large spacecraft can be equipped with the life support system appropriate for a long interplanetary flight with a crew, taxi vehicles of reduced size are sufficient to ensure the connection between the interplanetary vehicle and each planet. This research addresses the determination of the optimal low-thrust transfer trajectories for a pair of taxi vehicles aimed at transferring a crew or an inert payload from and toward a large Earth-Mars cycling spacecraft. The problem is solved through the indirect heuristic method, which uses the necessary conditions for optimality together with a heuristic technique, i.e. the particle swarm algorithm. The use of the indirect heuristic approach for low-thrust paths may be affected by hypersensitivity with respect to the initial values of the costate. This phenomenon is mitigated through a judicious choice of the equations of motion that govern the spacecraft dynamics. The solution method proposed in this work proves to be effective and very accurate, and leads to determining the minimum-time low-thrust paths for a pair of taxi vehicles traveling from and toward a large cycling spacecraft.

Earth-Mars transfers through Moon Distant Retrograde Orbits

This paper focuses on the trajectory design which is relevant for missions that would exploit the use of asteroid mining in stable cis-lunar orbits to facilitate deep space missions, specifically human Mars exploration. Assuming that a refueling “gas station” is present at a given lunar Distant Retrograde Orbit (DRO), ways of departing from the Earth to Mars via that DRO are analyzed. Thus, the analysis and results presented in this paper add a new cis-lunar departure orbit for Earth-Mars missions. Porkchop plots depicting the required C3 at launch, v∞ at arrival, Time of Flight (TOF), and total ΔV for various DRO departure and Mars arrival dates are created and compared with results obtained for low ΔV Low Earth Orbit (LEO) to Mars trajectories. The results show that propellant-optimal trajectories from LEO to Mars through a DRO have higher overall mission ΔV due to the additional stop at the DRO. However, they have lower Initial Mass in LEO (IMLEO) and thus lower gear ratio as well as lower TOF than direct LEO to Mars transfers. This results in a lower overall spacecraft dry mass that needs to be launched into space from Earth's surface.

A Novel Model for the Optimization of Interplanetary Trajectory Using Evolutionary Algorithm

A novel model, called evolutionary patched model, based on the patched conic approximation is applied to the optimization of space missions with engineering constraints in this paper. The interplanetary trajectory consists of geocentric escape orbit, heliocentric transfer orbit and target capture orbit. The model, firstly, gets the escape orbit and capture orbit by optimizing the elements of orbits with evolutionary algorithm, and then calculate the heliocentric transfer orbit by solving the Lambert’s problem, which is just opposite to the procedure of patched conic method. A module based on orthogonal test is introduced in the initialization process to produce a good starting population for the evolutionary algorithm. Earth-Mars mission has been considered. The results show that the good performance obtained with this mode and the improvement in terms of efficiency and computational cost.

Cooperative evolutionary algorithm for space trajectory optimization

A hybrid evolutionary algorithm which synergistically exploits differential evolution, genetic algorithms and particle swarm optimization, has been developed and applied to spacecraft trajectory optimization. The cooperative procedure runs the three basic algorithms in parallel, while letting the best individuals migrate to the other populations at prescribed intervals. Rendezvous problems and round-trip Earth–Mars missions have been considered. The results show that the hybrid algorithm has better performance compared to the basic algorithms that are employed. In particular, for the rendezvous problem, a 100% efficiency can be obtained both by differential evolution and the genetic algorithm only when particular strategies and parameter settings are adopted. On the other hand, the hybrid algorithm always attains the global optimum, even though nonoptimal strategies and parameter settings are adopted. Also the number of function evaluations, which must be performed to attain the optimum, is reduced when the hybrid algorithm is used. In the case of Earth–Mars missions, the hybrid algorithm is successfully employed to determine mission opportunities in a large search space.

Earth–Mars halo to halo low thrust manifold transfers

Application of low thrust propulsion to interconnect ballistic trajectories on invariant manifolds associated with multiple circular restricted three body systems has been investigated. Sun-planet three body models have been coupled to compute the two ballistic trajectories, where electric propulsion is used to interconnect these trajectories as no direct intersection in the Poincare sections exists. The ability of a low thrust to provide the energy change required to transit the spacecraft between two systems has been assessed for some Earth to Mars transfers. The approach followed consists in a planetary escape on the unstable manifold starting from a periodic orbit around one of the two collinear libration points near the secondary body. Following the planetary escape and the subsequent coasting phase, the electric thruster is activated and executes an ad-hoc thrusting phase. The complete transfer design, composed of the three discussed phases, and possible applications to Earth–Mars missions is developed where the results are outlined in this paper.

Computation of optimal Mars trajectories via combined chemical/electrical propulsion, Part 3: Compromise solutions

The success of the solar-electric ion engine powering the DS1 spacecraft has paved the way toward the use of low-thrust electrical engines in future planetary/interplanetary missions. Vis-à-vis a chemical engine, an electrical engine has a higher specific impulse, implying a possible decrease in propellant mass; however, the low-thrust aspect discourages the use of an electrical engine in the near-planet phases of a trip, since this might result in an increase in flight time. Therefore, a fundamental design problem is to find the best combination of chemical propulsion and electrical propulsion for a given mission, for example, a mission from Earth to Mars. With this in mind, this paper is the third of a series dealing with the optimization of Earth–Mars missions via the use of hybrid engines, namely the combination of high-thrust chemical engines for planetary flight and low-thrust electrical engines for interplanetary flight. We look at the deep-space interplanetary portion of the trajectory under rather idealized conditions. The two major performance indexes, the propellant mass and the flight time, are in conflict with one another for the following reason: any attempt at reducing the former causes an increase in the latter and vice versa. Therefore, it is natural to consider a compromise performance index involving the scaled values of the propellant mass and flight time weighted respectively by the compromise factor C and its complement 1 - C . We use the compromise factor as the parameter of the one-parameter family of compromise trajectories. Analyses carried out with the sequential gradient-restoration algorithm for optimal control problems lead to results which can be highlighted as follows. Thrust profile. Generally speaking, the thrust profile of the compromise trajectory includes three subarcs: the first subarc is characterized by maximum thrust in conjunction with positive (upward) thrust direction; the second subarc is characterized by zero thrust (coasting flight); the third subarc is characterized by maximum thrust in conjunction with negative (downward) thrust direction. Effect of the compromise factor. As the compromise factor increases, the propellant mass decreases and the flight time increases; correspondingly, the following changes in the thrust profile take place: (a) the time lengths of the first and third subarcs (powered phases) decrease slightly, meaning that thrust application occurs for shorter duration; also, the average value of the thrust direction in the first and third subarcs decreases, implying higher efficiency of thrust application wrt the spacecraft energy level; as a result, the total propellant mass decreases; (b) the time length of the second subarc (coasting) increases considerably, resulting in total time increase. Minimum time trajectory. If C = 0 , the resulting minimum time trajectory has the following characteristics: (a) the time length of the coasting subarc reduces to zero and the three-subarc trajectory degenerates into a two-subarc trajectory; (b) maximum thrust is applied at all times and the thrust direction switches from upward to downward at midcourse. Minimum propellant mass trajectory. If C = 1 , the resulting minimum propellant mass trajectory has the following characteristics: (a) the thrust magnitude has a bang-zero-bang profile; (b) for the powered subarcs, the thrust direction is tangent to the flight path at all times.

Computation of optimal Mars trajectories via combined chemical/electrical propulsion, Part 2: Minimum time solutions with bounded thrust direction

The success of the solar-electric ion engine powering the DS1 spacecraft has paved the way toward the use of low-thrust electrical engines in future planetary/interplanetary missions. Vis-à-vis a chemical engine, an electrical engine has a higher specific impulse, implying a decrease in propellant mass; however, the low-thrust aspect discourages the use of an electrical engine in the near-planet phases of a trip, since this might result in an increase in flight time. Therefore, a fundamental design problem is to find the best combination of chemical propulsion and electrical propulsion for a given mission, for example, a mission from Earth to Mars. With this in mind, this paper is the second of a series dealing with the optimization of Earth–Mars missions via the use of hybrid engines, namely the combination of high-thrust chemical engines for planetary flight and low-thrust electrical engines for interplanetary flight. We look at the deep-space interplanetary portion of the trajectory under rather idealized conditions. We study minimum time trajectories with bounded thrust direction and bounded thrust magnitude: the thrust direction α is subject to the inequality - α max ⩽ α ⩽ α max , while the thrust setting β is subject to the inequality 0 ⩽ β ⩽ 1 . We use α max as a parameter to generate a family of minimum time trajectories. Numerical results show that, as α max decreases, the flight time increases, while the propellant consumption decreases. Generally speaking, the thrust profile of the optimal trajectory includes three subarcs: the first subarc is characterized by maximum thrust in conjunction with positive (upward) thrust direction; the second subarc is characterized by zero thrust (coasting flight, thrust direction irrelevant); the third subarc is characterized by maximum thrust in conjunction with negative (downward) thrust direction. Two limiting cases have a particular interest. The first limiting case occurs for α max = 180 ∘ and has the following properties: (i) the time length of the coasting subarc reduces to zero and the three-subarc trajectory degenerates into a two-subarc trajectory; (ii) maximum thrust is applied at all times and the thrust direction switches from positive to negative just beyond midway; (iii) the minimum time trajectory for α constrained is the same as the minimum time trajectory for α unconstrained. The second limiting case occurs for α max = 0 ∘ and has the following properties: (i) the thrust magnitude has a bang-zero-bang profile; (ii) for the powered subarcs, the thrust direction is tangent to the flight path at all times; (iii) the minimum time trajectory for α constrained is the same as the minimum propellant consumption trajectory for α unconstrained.

Computation of optimal Mars trajectories via combined chemical/electrical propulsion, part 1: baseline solutions for deep interplanetary space

The success of the solar-electric ion engine powering the DS1 spacecraft has paved the way toward the use of low-thrust electrical engines in future planetary/interplanetary missions. Vis-à-vis a chemical engine, an electrical engine has a higher specific impulse, implying a decrease in propellant mass; however, the low-thrust aspect discourages the use of an electrical engine in the near-planet phases of a trip, since this might result in an increase in flight time. Therefore, a fundamental design problem is to find the best combination of chemical propulsion and electrical propulsion for a given mission, for example a mission from Earth to Mars. With this in mind, this paper is the first of a series dealing with the optimization of Earth–Mars missions via the use of hybrid engines, namely the combination of high-thrust chemical engines for planetary flight and low-thrust electrical engines for interplanetary flight. We look at the deep-space interplanetary portion of the trajectory under rather idealized conditions. The direct objective is to minimize the propellant consumption while containing the flight time; the indirect objective is to generate a nominal trajectory of easy implementation for guidance. We study the trajectory of a spacecraft controlled via the thrust direction and magnitude. We consider four control models. In Model 1, the thrust magnitude is set at the maximum value and the thrust direction is optimized. This results in a continuously varying thrust direction and a two-subarc trajectory. While the tangential component of the thrust is mostly directed forward, the normal component is directed upward in the first subarc and downward in the second subarc. The switch from the first subarc to the second is needed to shorten the flight time, albeit at the expense of higher propellant consumption. In Model 2, the thrust direction is assumed tangent to the trajectory and the thrust magnitude is optimized. This results in a three-subarc trajectory having a bang-zero-bang thrust magnitude profile, of particular interest in guidance because of its simplicity. Vis-à-vis Model 1, the trajectory optimizing Model 2 exhibits a significant decrease in propellant consumption, albeit at the expense of an increase in flight time. In Model 3, the thrust direction and magnitude are simultaneously optimized. For the minimum time problem, the solution of Model 3 reduces to that of Model 1. For the minimum propellant consumption problem, the solution of Model 3 reduces to that of Model 2. In Model 4, which is a discrete version of Model 2, the thrust direction is assumed tangent to the trajectory and the thrust magnitude is optimized in the class of controls allowed to have only two values: maximum thrust and zero thrust. The solution of this problem is identical with that of Model 2. For Models 1–3, the optimization problems were solved with the sequential gradient-restoration algorithm in either single-subarc form or multiple-subarc form. For Model 4, due to the specific problem structure, the solution was found with the modified quasilinearization algorithm.

Minimum fuel control of linear stochastic systems with applications to midcourse guidance

The general problem of minimum fuel control of discrete time gaussian processes is examined. Necessary conditions are derived for the optimal control of systems with both process and measurement uncertainties. A fixed correction time strategy is studied and results are applied to a spacecraft midcourse guidance problem. A variable correction time strategy is also treated and it is shown that the fixed time solution is useful in determining the optimal variable correction time control. Numerical results are presented for an Earth-Mars mission employing both fixed and variable correction time strategies.