In this correspondence paper, an image dynamics-based visual servoing for quadrotors is proposed to realize stable hovering and tracking. Four perspective image moments are adopted as visual features to control all the independent degrees of freedom of a quadrotor. The complicated interaction matrix is simplified by projecting original image to virtual image plane. On this basis, the dynamics of the system is determined by considering the dynamics of image features and the quadrotor simultaneously. Backstepping controllers are then designed to stabilize the visual servoing system of the quadrotor. In reality, it is unrealistic to have exact prior knowledge about the trajectory parameters of an unpredictable moving target. To solve this problem, a trajectory observer based on nonlinear tracking-differentiator to estimate trajectory parameters of the target is firstly integrated into the quadrotor with image dynamics, which guarantees a satisfactory performance. The effectiveness of the proposed approach is verified by simulations.