The inconsistency between experiments in the measurements of the local Universe expansion rate, the Hubble constant, suggests unknown systematics in the existing experiments or new physics. Gravitational-wave standard sirens, a method to independently provide direct measurements of the Hubble constant, have the potential to address this tension. Before that, it is critical to ensure there are no substantial systematics in the standard siren method. A significant systematic has been identified when the viewing angle of the gravitational-wave sources, the compact binary coalescences, was inferred inaccurately from electromagnetic observations of the sources. Such a systematic has led to a more than 10% discrepancy in the standard siren Hubble constant measurements with the observations of binary neutron star merger, GW170817. In this Letter, we develop a new formalism to infer and mitigate this systematic. We demonstrate that the systematic uncertainty of the Hubble constant measurements can be reduced to a level smaller than their statistical uncertainty with 5, 10, and 20 binary neutron star merger observations. We show that our formalism successfully reduces the systematics even if the shape of the biased viewing angle distribution does not follow precisely the model we choose. Our formalism ensures unbiased standard siren Hubble constant measurements when the binary viewing angles are inferred from electromagnetic observations.