This paper presents a control scheme for standoff tracking of a ground moving target by a quadrotor unmanned aerial vehicle (UAV). The control system is decoupled into outer loop for position control and inner loop for attitude regulation. In the outer loop design, the standoff motion of the vehicle is described in a cylindrical coordinate system attached to the target. After that, the standoff tracking guidance law is designed based on a Lyapunov potential function which can guarantee the stability of the movement. The acceleration signals are produced from the proposed guidance law, and converted to Euler angle commands for the inner control system. A integral backstepping controller is developed to stabilize the attitude of the quadrotor. In particular, the disturbance observer technique is used to deal with the correction terms that account for a non-uniform moving target and constant wind. Numerical simulations are performed to verify the feasibility and performance of the proposed control scheme.