We investigate the irreversible adsorption of poly(3-hexylthiophene) (P3HT) polymer thin films on silicon dioxide/silicon (SiO2/Si) substrates during thermal annealing at a temperature below the melting temperature (Tm) but far above the glass transition temperature (Tg), i.e., Tg ≪ T = 170 °C < Tm, and its effect on their crystalline ordering and charge transport properties. It was found that short-time annealing enhances the molecular ordering of P3HT films, while prolonged thermal annealing gradually disrupts the crystalline structures and reduces the overall crystallinity of the film. Concurrently, thermal annealing at this temperature facilitates the slow irreversible adsorption of P3HT chains at the polymer-solid interface, resulting in the formation of a 1.7 Rg-thick (∼18 nm thick) adsorbed layer on SiO2/Si substrates that is fully amorphous and contains a large fraction of loosely adsorbed chains. We postulate that such irreversible adsorption is responsible for the reduced crystalline packing of P3HT at the polymer-solid interface at Tg ≪ T < Tm, which further disrupts the molecular ordering of the entire 46 nm thick P3HT film by a long-range perturbation effect. Electrical measurements using an organic field-effect transistor (OFET) device reveal that the enhanced charge carrier mobility of P3HT films correlates with an optimized annealing time at Tg ≪ T < Tm, which achieves a balance between maximizing molecular ordering and minimizing the impact of irreversible chain adsorption. These findings provide new insights into the underlying mechanism of thermal annealing in tailoring the structure and property of conjugated polymer thin films prepared on solid substrates.