We study the effect of ultra-thin oxide (SiOx) layers inserted at the interfaces of silicon heterojunction (SHJ) solar cells on their open-circuit voltage (VOC). The SiOx layers can be easily formed by dipping c-Si into oxidant such as hydrogen peroxide (H2O2) and nitric acid (HNO3). We confirm the prevention of the undesirable epitaxial growth of Si layers during the deposition of a-Si films by the insertion of the ultra-thin SiOx layers. The formation of the SiOx layers by H2O2 leads to better effective minority carrier lifetime (τeff) and VOC than the case of using HNO3. c-Si with the ultra-thin SiOx layers formed by H2O2 dipping, prior to deposition of a-Si passivation layers, can have high implied VOC of up to ∼0.714 V.