To enhance the long-term localization accuracy of inertial measurement unit/time-differenced carrier phase (IMU/TDCP) tightly coupled navigation system (TCS), efforts from three aspects have been made in this article. First, we make a comprehensive analysis, which has not been done before, on the errors of TDCP measurement. Then, these errors are modeled and mostly compensated using the polynomial fitting method. Even so, the errors cannot be eliminated completely. Positioning performance will get worse slowly over time due to the error remained in the error-compensated TDCP (CTDCP). Therefore, the final effort has been done in introducing a novel measurement, ionosphere-free wide-lane carrier phase (IFWLCP), to control the deterioration instead of the conventional coarse pseudorange (PR). The CTDCP and IFWLCP are both integrated with IMU measurements using the dual-rate Kalman filter, and the proposed methodology’s effectiveness on improving the positioning accuracy is validated by a road experiment.