Augmented reality using optical see-through head-mounted displays (OSTHMDs) provides the user with a highly realistic experience compared to those using smartphones or tablet devices. It is necessary for the positional relationship between the user's eye and a virtual screen to be calibrated using input from the user. However, conventional calibration methods are highly sensitive to input errors. In this paper, we propose a vision-based robust calibration (ViRC) method using a fiducial marker, which can be used for any OSTHMD equipped with a camera. The ViRC method decomposes 11-DoFs calibration parameters into device-dependent parameters and user-dependent parameters. Once the device-dependent parameters are calculated, the user only has to perform a calibration phase for estimating the 4-DoFs user-dependent parameters. Experiments show that the ViRC method can decrease reprojection error by 83% compared with the conventional method. Consequently, users can observe correctly aligned superimpositions of computer graphics with little distortion.