Recently, a considerable amount of attention has been given to the search for ultralight dark matter by measuring the oscillating length changes in the arm cavities of gravitational wave detectors. Although gravitational wave detectors are extremely sensitive for measuring the differential arm length changes, the sensitivity to dark matter is largely attenuated, as the effect of dark matter is mostly common to arm cavity test masses. Here, we propose to use auxiliary length channels, which measure the changes in the power and signal recycling cavity lengths and the differential Michelson interferometer length. The sensitivity to dark matter can be enhanced by exploiting the fact that auxiliary interferometers are more asymmetric than two arm cavities. We show that the sensitivity to $U(1)_{B-L}$ gauge boson dark matter with masses below $7\times 10^{-14}$ eV can be greatly enhanced when our method is applied to a cryogenic gravitational wave detector KAGRA, which employs sapphire test masses and fused silica auxiliary mirrors. We show that KAGRA can probe more than an order of magnitude of unexplored parameter space at masses around $1.5 \times 10^{-14}$ eV, without any modifications to the existing interferometer.