The measurement of a model ship's six degrees of freedom (6-DOF) using sensors such as Marvelmind beacons and robotic total stations (RTS) has previously demonstrated substantial error margins. To address these limitations, this study introduces a refined approach for accurate and reliable 6-DOF measurement through free-running model tests (FRMT). The tests were meticulously designed, with three RTS strategically positioned to meet the specific requirements of the measurement scenario. Given the potential constraints in an FRMT environment, we proposed feasible installation ranges for the RTS and prisms, ensuring the stable measurement of the model ship's 6-DOF. To achieve high-precision motion measurement, even in complex environments, we employed the prism queue algorithm, which identifies and processes overlapping or untracked prisms monitored by the RTS. The proposed methodology was validated through simulations and FRMT conducted in the ocean engineering basin of the Korea Research Institute of Ships and Ocean Engineering and by developing an actual implementation program. Comparative analysis using high-precision navigation equipment, such as fiber optic gyros, demonstrated that the method provided unbiased and stable data. Consequently, the proposed method shows significant promise as a viable alternative for precision navigation in real-world applications.