Quadrotor unmanned aerial vehicles (UAVs) have emerged as versatile platforms applicable to various fields such as surveillance, reconnaissance, and mapping. However, the limited energy capacity inherent in these vehicles poses a significant challenge. This limitation is a critical concern hindering their widespread adoption across diverse applications. Despite various proposed technological solutions addressing the energy consumption issue, the central challenge for the control community lies in developing controllers that ensure stability, robustness, and energy efficiency. In this paper, we present a passivity-based sliding mode controller (P-SMC) designed specifically for quadrotor UAVs. The suggested controller capitalizes on the robustness of the SMC and the energy efficiency of passivity-based control theory. Feedback passification is employed to create a sliding surface with passivity, ensuring stability. The inclusion of a noncontinuity term in the proposed P-SMC guarantees global asymptotic convergence to the sliding surface. To address the multi-faceted nature of the problem, a multi-objective optimization criterion is introduced. This criterion, which combines tracking error and control energy, is solved using the ant colony optimization (ACO) algorithm. The optimization process aims to identify control parameters that strike a balance between tracking accuracy and energy efficiency. Additionally, a conventional sliding mode method is developed to respond to conditions of attempted reach and sliding. Finally, the effectiveness of the proposed method is demonstrated through simulation results considering piecewise constant external disturbances. The outcomes based on the proposed control strategy indicate a notable reduction in energy consumption (ranging from [Formula: see text] to [Formula: see text]), faster reaching times (5–8 times), higher accuracy approximately ([Formula: see text]), and reduced chattering compared to the conventional sliding mode technique.