Polarization-sensitive organic photodetectors (OPDs) are increasingly paid attention for their broad application prospect. However, it is extremely difficult to achieve highly sensitive detection of polarized light for most organic semiconductor polymers with intrinsic isotropic amorphous behavior. In this study, the performance-enhanced intrinsic anisotropic all-polymer bulk heterojunctions (BHJs) are obtained by constructing molecular fluorination engineering in the fused-ring backbone of the acceptor polymer. The molecular orientation, alignment, and packing can be effectively optimized during the floating-film formation process without using any additives and post-treatments. The outstanding intrinsic polarization-sensitive photodetection performance with a very high photocurrent dichroic ratio of 3.73, a specific detectivity of 1.3 × 1011 Jones at 0 V, and a broad linear dynamic range of 120 dB is achieved for the optimum-fluorine-content self-assembly all-polymer organic BHJ film. The superiorities of self-powered capability, fast response, and high-contrast imaging demonstrate that the backbone fluorination for acceptor polymers is conducive to achieving excellent structural regularity and can endow the intrinsic polarization-sensitive all-polymer OPDs with more demanding polarized-light detection ability.