A modified cubature Kalman filter (CKF) algorithm is proposed for an in-pipe survey system. This survey system can provide accurate three-dimensional (3D) location information for underground pipelines without external auxiliary location measurements. To move through a small-diameter pipe, the large size of the tactical grade inertial sensor is replaced with a low-cost micro-electromechanical system (MEMS) inertial measurement unit (IMU). Because the decrease in sensor accuracy increases the positioning error, it is necessary to improve the original filtering algorithm. The following improvements were made in relation to previous studies: geomagnetic and gravity sensors were introduced to obtain the observed vectors of the filter. The CKF was used to solve the nonlinearity of the filter's attitude errors. Furthermore, the CKF process noise matrix can be adaptively adjusted using the raw gyroscope measurements (ACKF) to maintain filter stability. Finally, the experimental results for the extended Kalman filter (EKF), unscented Kalman filter (UKF), CKF, and ACKF were compared to verify the effectiveness of ACKF.