The effects of interface roughness between donor and acceptor in a bilayer heterojunction solar cell were investigated on a polymer–polymer system based on poly(3-hexylthiophene) (P3HT) and poly(dioctylfluorene-alt-benzothiadiazole) (F8BT). Both polymers are known to reorganize into semicrystalline structures when heated above their glass-transition temperature. Here, the bilayers were thermally annealed below glass transition of the bulk polymers (≈140 °C) at temperatures of 90, 100, and 110 °C for time periods from 2 min up to 250 min. No change of crystallinity could be observed at those temperatures. However, X-ray reflectivity and device characteristics reveal a coherent trend upon heat treatment. In X-ray reflectivity investigations, an increasing interface roughness between the two polymers is observed as a function of temperature and annealing time, up to a value of 1 nm. Simultaneously, according bilayer devices show an up to 80% increase of power conversion efficiency (PCE) for short annealing periods at any of the mentioned temperatures. Together, this is in agreement with the expectations for enlargement of the interfacial area. However, for longer annealing times, a decrease of PCE is observed, despite the ongoing increase of interface roughness. The onset of decreasing PCE shifts to shorter durations the higher the annealing temperature. Both, X-ray reflectivity and device characteristics display a significant change at temperatures below the glass transition temperatures of P3HT and F8BT.