Increasing the active layer thickness without sacrificing the power conversion efficiency (PCE) is one of the great challenges faced by organic solar cells (OSCs) for commercialization. Recently, PM6:Y6 as an OSC based on a non‐fullerene acceptor (NFA) has excited the community because of its PCE reaching as high as 15.9%; however, by increasing the thickness, the PCE drops due to the reduction of the fill factor (FF). This drop is attributed to change in mobility ratio with increasing thickness. Furthermore, this work demonstrates that by regulating the packing and the crystallinity of the donor and the acceptor, through volumetric content of chloronaphthalene (CN) as a solvent additive, one can improve the FF of a thick PM6:Y6 device (≈400 nm) from 58% to 68% (PCE enhances from 12.2% to 14.4%). The data indicate that the origin of this enhancement is the reduction of the structural and energetic disorders in the thick device with 1.5% CN compared with 0.5% CN. This correlates with improved electron and hole mobilities and a 50% suppressed bimolecular recombination, such that the non‐Langevin reduction factor is 180 times. This work reveals the role of disorder on the charge extraction and bimolecular recombination of NFA‐based OSCs.