Geolocation accuracy is a critical issue for remote sensing applications. To achieve subpixel accuracy, geolocation errors need to be systematically identified and corrected. In this study, we propose a geometric sensor model for FengYun-3D (FY-3D) MERSI-II, a second-generation visible (VIS)/infrared (IR) spectroradiometer, to generate the geolocation lookup table (GLT). The geometric sensor model retrieves the imaging rays from the focal plane to the K-mirrors, 45° scanning mirrors, the platform, and the earth’s surface. After refining the attitude errors with ground control points (GCPs), the rigorous sensor model can achieve subpixel geolocation accuracy. However, significant systematic geolocation errors were identified from the residuals, especially for the area with large view angles. To study the errors of MERSI-II, we proposed a homogenous coordinate in the focal plane. As proven by both theory and experiments, the attitudes were adjusted to a wrong value and introduced systematic errors when there were principal point errors. The pitch angle error of K-mirrors caused the oscillation in the flight direction. The principal distance error introduced line coordinate-related error in the flight direction. Meanwhile, the initial phase angle error between the K-mirror and 45° scanning mirrors caused the line coordinate-related errors in the scanning direction. After correcting all the above-mentioned errors, the systematic geolocation errors of MERSI-II were removed. With 23 independent datasets, the root mean square errors (RMSEs) of 250 m bands were approximately 0.4 pixels, 100 m at nadir.