We have developed an attitude-independent calibration method for a shipboard magnetometer to estimate the absolute strength of the geomagnetic field from a marine vessel. The three-axis magnetometer to be calibrated is fixed on a rigid aluminium boom ahead of the vessel to reduce the magnetic effect of the vessel. Due to the constrained manoeuvres of the vessel, a linear observational equation system for calibration parameter estimation is severely ill-posed. Consequently, if the issue is not mitigated, traditional calibration methods may result in unreliable or unsuccessful solutions. In this paper, the ill-posed problem is solved by using the truncated total least squares (TTLS) technique. This method takes advantage of simultaneously considering errors on both sides of the observation equation. Furthermore, the TTLS method suits strongly ill-posed problems. Simulations and experiments have been performed to assess the performance of the TTLS method and to compare it with the performance of conventional regularization approaches such as the Tikhonov method and truncated single value decomposition. The results show that the proposed algorithm can effectively mitigate the ill-posed problem and is more stable than the compared regularization methods for magnetometer calibration applications.