The 2.5-generation (2.5G) ground-based gravitational wave (GW) detectors LIGO Voyager and NEMO are expected to be operational in the late 2020s and early 2030s. In this work, we explore the potential of GW standard sirens observed by the 2.5G GW detectors in measuring cosmological parameters, especially for the Hubble constant. Using GWs to measure cosmological parameters is inherently challenging, especially for 2.5G detectors, given their limited capability, which results in weaker constraints on cosmological parameters from the detected standard sirens. However, the measurement of the Hubble constant using standard siren observations from Voyager and NEMO is still promising. For example, using bright sirens from Voyager and NEMO can measure the Hubble constant with a precision of about 2% and 6% respectively, and using the Voyager-NEMO network can improve the precision to about 1.6%. Moreover, bright sirens can be used to break the degeneracy of cosmological parameters generated by CMB data, and to a certain extent, 2.5G detectors can also play a role in this aspect. Observations of dark sirens by 2.5G detectors can achieve relatively good results in measuring the Hubble constant, with a precision of within 2%, and if combining observations of bright and dark sirens, the precision of the Hubble constant measurement can reach about 1.4%. Finally, we also discussed the impact of the uncertainty in the binary neutron star merger rate on the estimation of cosmological parameters. We conclude that the magnificent prospect for solving the Hubble tension is worth expecting in the era of the 2.5G ground-based GW detectors.