This paper proposes a robust hierarchical controller using adaptive radical basis function neural networks (RBFNNs) based proportional derivative-sliding mode control (RPD-SMC) method and robust integral of the signum of error (RISE) approach for an under-actuated quadrotor in the presence of disturbances and parametric uncertainties. The quadrotor system is decoupled into two parts: the outer loop for position control and the inner loop for attitude control. The RPD-SMC is designed for the outer loop to ensure robust position tracking. The PRD-SMC combines the advantages of simplicity of PD control, the strong robustness of SMC and the approximation ability of arbitrary functions of RBFNNs. The RISE method is applied in the inner loop to guarantee fast convergence of the attitude angles to their desired values with continuous control signals. The capabilities of online approximating and null steady-state tracking are proved using Lyapunov stability theory. The effectiveness of the proposed control strategy is validated by comparing with the performances achieved by PD, PID, PD-SMC and RBFNNs based controllers via numerical simulations.