The inertial measurement method of pipelines utilizes a Micro-Electro-Mechanical Systems Inertial Measurement Unit (MIMU) to get the three-dimensional trajectory of underground pipelines. The initial attitude is significant for the inertial measurement method of pipelines. The traditional method to obtain the initial attitude uses three-axis magnetometers to measure the Earth's magnetic field. However, the magnetic field in urban underground pipelines is intricate, which leads to the initial attitude being inaccurate. To overcome this challenge, a novel multi-position initial alignment method based on data backtracking for a single-axis FOG and a three-axis Micro-Electro-Mechanical Inertial Measurement Unit (MIMU) is proposed. Firstly, the configuration of the sensors is determined. Secondly, according to the three-point support structure of the pipeline measuring instrument, a three-position alignment scheme is designed. Additionally, an initial alignment algorithm using the data backtracking method is developed. In this algorithm, a rough initial alignment is conducted by the data from single-axis FOG, and a fine initial alignment is conducted by the data from FOG/MIMU. Finally, an experiment was conducted to validate this method. The experiment results indicate that the pitch and roll angle errors are less than 0.05°, and the azimuth angle errors are less than 0.2°. This improved the precision of the 3-D trajectory of underground pipelines.