The integration of magnetometers and Inertial Navigation Systems (INS) is widely used in low-cost navigation systems. However, even if the system has been calibrated, random magnetic disturbances still appear in practical applications, which lead to large heading errors. To solve this problem, an adaptive anti-disturbance method to overcome random magnetic disturbance is proposed. First, disturbances are classified and analysed in detail based on actual road vehicle driving data. Then an Adaptive Robust Extend Kalman Filter (AREKF) is designed to resist sudden disturbances. However, an AREKF may accumulate errors slowly when a long-term disturbance exists. Considering this situation, this paper proposes that AREKF is used to maintain accuracy in the early stages, at the same time as the magnetometer is quickly calibrated with a Kalman filter. Then, the new magnetometer parameters are put into the AREKF to suppress long-term disturbances. Finally, cascading these two modules, not only the sudden disturbance can be overcome, but the situation of long-term disturbances can be suppressed. The results of simulation and an actual driving test show that the proposed method can effectively overcome random magnetic disturbances in both the short and long term.