All-polymer photovoltaic devices have attracted extensive attention for their promising applications in various flexible devices. Although abundant polymer donors with superior properties are developed, new polymer acceptors are urgently required to optimize or update the performances of the devices. Herein, three novel polymer acceptors, namely PY-DTT, PY-DTP-C, and PY-DTP-Ph, based on dithienothiophene (DTT) and dithienopyrrole (DTP) with alkyl and aryl side chains as the linking units respectively are synthesized and employed in the all-polymer photodetectors (all-PPDs). The PY-DTP-Ph shows broader absorption, stronger intermolecular interaction, and more coplanar molecular conformation compared with the other two acceptors. Blending with donor PM6, the PY-DTP-Ph based all-PPDs exhibit maximum responsivity of 292.1 mA W−1 and specific detectivity of 5.4 × 1013 cm Hz1/2 W−1 at a bias of −0.1 V. Furthermore, the device achieves fast response speed (τr/τf = 1.2/21.8 μs) and a large linear dynamic range of 105.0 dB. Intriguingly, the photodetectors present extraordinary air stability for 5000 h, attributing to the ideal miscibility and morphology of the blend films. These results demonstrate close relationships between the electron linking units, side chains and the basic properties of polymer acceptors, which in turn affects the performances of the photodetector devices.