Temperature-dependent aggregation is a key property for some donor polymers to realize favorable bulk-heterojunction (BHJ) morphologies and high-efficiency (>10%) polymer solar cells. Previous studies find that an important structural feature that enables such temperature-dependent aggregation property is the 2nd position branched alkyl chains sitting between two thiophene units. In this report, we demonstrate that an optimal extent of fluorination on the polymer backbone is a second essential structural feature that enables the strong temperature-dependent aggregation property. We compare the properties of three structurally similar polymers with 0, 2 or 4 fluorine substitutions in each repeating unit through an in-depth morphological study. We show that the non-fluorinated polymer does not aggregate in solution (0.02 mg mL−1 in chlorobenzene) at room temperature, which results in poor polymer crystallinity and extremely large polymer domains. On the other hand, the polymer with four fluorine atoms in each repeating unit exhibits an excessively strong tendency to aggregate, which makes it difficult to process and causes a large domain. Only the polymer with two fluorine atoms in each repeating unit exhibits a suitable extent of temperature-dependent aggregation property. As a result, its blend film achieves a favorable morphology and high power conversion efficiency. This provides another key design rationale for developing donor polymers with suitable temperature-dependent aggregation properties and thus high performance.