Today, GPS-free drone localization is increasingly gaining attention in various applications, but it faces significant accuracy challenges in three-dimensional (3D) space due to various impairments. This study investigates the effects of carrier frequency offset (CFO), phase noise (PN), and down-tilted base station (BS) antennas on drone positioning and tracking. Additionally, we explore the impact of inter-site distance (ISD) and BS density on drone position estimation accuracy. In our methodology, we consider a flying drone equipped with a single transmission antenna and BSs configured with 4 × 4 antennas under specific impairments. We first analyze the effects of these impairments on the signal’s covariance matrix. Then, using the MUSIC algorithm, we estimate the azimuth and elevation angles, which serve as the basis for drone localization using the Least Squares (LS) method across all BSs. Finally, the estimated positions feed into an Extended Kalman Filter (EKF) for tracking. Our results present a sequential analysis of the impact of all impairments on the off-diagonal covariance matrix, on the Angle of Arrival (AOA) estimation and 3D drone localization. We use simulations to demonstrate how hardware impairments affect 3D drone localization accuracy under varying ISD and BS densities.