Terpolymer fabrication is an effective methodology for molecular engineering and generating high-performance organic photovoltaic materials to construct highly efficient polymer solar cells. Modification of the polymer PM6 by incorporating a third component resulting in the formation of a ternary copolymer is reported to outperform PM6 in achieving enhanced device performances. However, one of the major challenges in constructing high-performance terpolymers is to counter the molecular disorder caused by the backbone entropy induced by the third moiety. In this work, double B←N bridged bipyridine (BNBP) is used as the third component, which possesses astrong out-of-plane electrostatic dipole owing to the saddle-shaped B←N fused ring structure. The out-of-plane dipole moment introduced in the modified PM6 terpolymer can be used as a means for tuning and optimizing the nanostructures of the blended films. The prepared PM6-BNBP-4 blend polymer with 4% of the benzodithiophene dione monomers replaced by BNBP results in excellent power conversion efficiency of 19.13%. This work demonstrates that the out-of-plane electrostatic dipole moment in saddle-shaped molecules is valuable for achieving high-performance organic photovoltaic donor materials.