In this article, a novel robust initial alignment method for the strapdown inertial navigation system (SINS) and global positioning system (GPS) integrated system is proposed to eliminate the outliers containing in the outputs of GPS. There are two innovations in the proposed method. First, the traditional quaternion Kalman filter method is often used to estimate the constant attitude quaternion in the swaying base. However, this article devises the new quaternion state model, which can be used in the in-motion base, expanding the scope of application. Second, based on the Huber's robust theory and the in-motion quaternion Kalman filter state model, the novel algorithm called the robust quaternion Kalman filter method is devised. This method can construct the improved observation vectors to eliminate the outliers and estimate the constant attitude quaternion more accurately than the traditional in-motion method when GPS contains outliers. The experiment using the miniature inertial measurement unit (MIMU)-based SINS aided by GPS has shown the effectiveness of the proposed method.