The strapdown magnetic vector measurement system, which can measure the magnetic vector and the attitude of a magnetometer simultaneously, has wide applications in geophysical prospecting, etc. Calibration of systematic errors, including magnetometer errors and misalignment errors, is essential for this system. Traditional methods calibrate these two errors separately, with the problem of cumbersome steps and being dependent on special data acquisition methods, such as rotation. An original method that combines a plane compression method with an ellipsoid fitting method is proposed in this paper, which can simultaneously complete the calibration calculation of magnetometer error and misalignment error in one experiment. The calculation can be performed using the spatial scatter point data required by the traditional attitude-independent magnetometer calibration method, and no additional mechanical equipment is required. A mathematical analysis of this method is performed to study the elements decreasing the measurement accuracy of the system, and numerical simulation and field experiments are performed to validate the analysis. The results indicate that the method can contribute to the accuracy improvement of magnetic vector measurement systems.