Conjugated polymers based on the donor of an asymmetric 5H-dithieno[3,2-b:2′,3′-d]pyran (DTPa) and the acceptors of benzo[d][1,2,3]triazole (BTA) or di-fluorinated benzo[d][1,2,3]triazole (ffBTA) with thiophene as π-bridge were designed and synthesized. Two asymmetric-building-block-containing polymers (ABC-polymers) possess a strong and broad absorption in the range of 300–750 nm and medium optical bandgap of 1.73 and 1.77 eV for PDTPa-TBTA and PDTPa-TffBTA, respectively. Polymer solar cells using PDTPa-TBTA as donor and [6,6]-phenyl-C71-butyric acid methyl ester (PC71BM) as an acceptor exhibited power conversion efficiencies (PCE) of 2.22% with a Voc of 0.58 V, a Jsc of 6.04 mA/cm2, and an FF of 63.41%. The introduction of fluorine substituents on the BTA unit evidently influenced the optical and photovoltaic properties. Interestingly, although the HOMO energy level indeed decreased, PDTPa-TffBTA showed a decreased Voc of 0.52 V in solar cells. Combined with an obviously enhanced Jsc of 10.23 mA/cm2, and an outstanding FF of 0.64, the PCE of solar cells based on PDTPa-TffBTA was improved by nearly 55%, reached 3.43%. Our results indicate that the BTA unit can be used to construct ABC polymers with a medium bandgap, and the introduction of fluorine on the BTA unit is also effective in improving the photovoltaic performance.