The traditional railway survey adopts a manual observation method, such as a total station measuring system. This method has high precision, but the amount of data is small, and the measurement efficiency is low. Manual measurement cannot meet the requirements of dynamic continuous high-precision holographic measurement during railway outages. Mobile laser scanning is a mobile mapping system based mainly on a laser scanner, inertial measurement unit (IMU) and panoramic camera. Mobile laser scanning has the advantages of high efficiency, high precision and automation. However, integrating inertial navigation data and mobile laser scanning data to obtain real 3D information about railways has always been an urgent problem to be solved. Therefore, a point cloud reconstruction method is proposed based on trajectory filtering for a mobile laser scanning system. This paper corrects the odometer data by identifying railway feature points through deep learning and uses Rauch–Tung–Striebel (RTS) filtering to optimize the trajectory results. Combined with the railway experimental track data, the maximum difference in the east and north coordinate direction can be controlled within 7 cm, and the average elevation error is 2.39 cm. This paper applies a multi-sensor integrated mobile detection system to railway detection. It is of great significance to the healthy development of the intelligent railway system.