NeuroDynamic Programming Approaches for Problems of Combined System-Design and Its Operational Management

Abstract

The paper considers a problem in which a new commercial transport aircraft is designed simultaneously with optimally allocating the new aircraft and existing aircraft of an airlines ∞eet to minimize cost while meeting passenger demand and trip balance constraints. This problem serves as an example of more a general problem: design of a new system along with design of the dynamic operational strategy in which the new system will be employed. The focus of this work is on allocating variable and flxed resources to topologically large-scale ∞ight routes over a flnite time horizon. The term variable resources indicates the inclusion of a yetto-be-designed aircraft, whose performance and cost characteristics are functions of aircraft design variables. Addressing the problem over a flnite time horizon allows for time-dependent aspects of passenger demand, aircraft availability and monetary value to be considered when determining the optimal aircraft design and allocation. As the problem dimension increases, the computational requirement for solving this type of problem becomes more expensive. Neuro-dynamic programming (NDP) is a recent methodology that can be used to approximately solve very large and complex stochastic decision and control problems. In this spirit, this paper studies the applicability of NDP algorithms to the airline example of a variable resource allocation problem over a flnite time horizon. Of primary interest is the ability of NDP to flnd approximate solutions to this computationally-expensive, dynamic, discrete-optimization problem as problem dimension increases. In this optimization over time, a policy constitutes a solution to allocation problem over flnite time horizon. Such a solution compromises the expense of flnding the actual optimal solution, while satisfying both time-independent and time-dependent constraints. This work, instead of flnding the best policy, adopts a rollout policy, a simplifled - yet powerful - NDP algorithm as an alternative to the best policy for the constrained discrete optimization problem over a flnite horizon. The use of the rollout policy as an approximate policy appears to ofier reasonable results with less computational efiort than an enumeration scheme.