Most Aeronautical and Astronautical Systems (AAS) are inherently complex, multidisciplinary, nonlinear, and computationally intensive for design and analysis. Utilizing the Reliability-Based Multidisciplinary Design Optimization framework can address the multidisciplinary nature of these systems while accounting for inherent uncertainties. In this paper, an efficient methodology for Reliability-Based Multidisciplinary Design optimization of an aerial vehicle is developed. The computational burden of reliability assessment could make its integration within a Multidisciplinary Design Optimization cycle a formidable task. In this respect, a multilevel Multidisciplinary Design Optimization architecture is proposed in which the computational cost is reduced by considering the reliability analysis, as needed only for critical subsystems. To this end, a single-level Reliability-Based Multidisciplinary Design Optimization is derived using the Performance Measure Analysis and the Karush-Kuhn-Tucker condition. The work demonstrates the integration of this formulation into the proposed multilevel Reliability-Based Multidisciplinary Design Optimization architecture. The proposed design architecture is implemented for an aeroelastic Unpowered Guided Aerial Vehicle whose outcomes are compared with previous results obtained via a mono-level Uncertainty-Based Multidisciplinary Design Optimization architecture.