Semiconductor-based heterostructure composites with high ionic conductivity have gained considerable attentions as potential electrolytes for low temperature solid oxide fuel cells (LT-SOFCs). Thereinto, the p-n junction is designed for the built-in electric field, thus avoiding short-circuiting occurrence and implementing fuel cell functionality. Herein, a novel strategy to construct the dual-layer electrolyte using p-type CuFeO2 (CFO) and n-type ZnO respectively as individual layer is proposed. Electrochemical studies find that the fuel cell performance is significantly correlated with the weight ratios of CFO and ZnO phases. The device based on 3CFO-7ZnO (CFO in 30 wt% and ZnO in 70 wt%) electrolyte exhibits an open circuit voltage (OCV) of 1.06 V and a peak power density (PPD) of 557 mW cm−2 at 550 °C, apparently superior to both of the direct mixture of CFO-ZnO nanocomposite (468 mW cm−2) and single-phase ZnO electrolytes (300 mW cm−2) at the same operating temperature. Further investigations on energy band structure and rectifying characteristics related to the CFO and n-type ZnO indicate the existence of p-n junction. This work proposed a new method to design effective dual-layer semiconductor-based electrolytes for LT-SOFC.