The integrated navigation of inertial navigation systems (INS) and the Global Positioning System (GPS) is essential for small unmanned aerial vehicles (UAVs) such as multicopters, providing steady and accurate position, velocity, and attitude information. Nevertheless, decreasing navigation accuracy is a serious threat to flight safety due to the long-term drift error of INS in the absence of GPS measurements. To bridge the GPS outage for multicopters, this paper proposes a novel navigation reconstruction method for small multicopters, which combines the vehicle dynamic model and micro-electro-mechanical system (MEMS) sensors. Firstly, an induced drag model is introduced into the dynamic model of the vehicle, and an efficient online parameter identification method is designed to estimate the model parameters quickly. Secondly, the body velocity can be calculated from the vehicle model and accelerometer measurement. In addition, the nongravitational acceleration estimated from body velocity and radar height are utilized to yield a more accurate attitude estimate. Fusing the information of the attitude, body velocity, magnetic heading, and radar height, a navigation system based on an error-state Kalman filter is reconstructed. Then, an adaptive measurement covariance algorithm based on a fuzzy logic system is designed to reduce the weight due to the disturbed acceleration. Finally, the hardware-in-loop experiment is carried out to demonstrate the effectiveness of the proposed method. Simulation results show that the proposed navigation reconstruction algorithm aided by the vehicle model can significantly improve navigation accuracy during a GPS outage.