The attraction between π-conjugated planar electron donor and acceptor molecules that form many stable charge-transfer (CT) complexes has been explained by quantum chemical CT interactions, although the fundamental origin remains unclear. Here, we demonstrate the mechanism of CT complex formation by potential energy map analysis for TTF-CA and BTBT-TCNQ, using energy decomposition of intermolecular interaction by symmetry-adapted perturbation theory (SAPT) combined with coupled cluster calculation. We find that the source of attraction between donor and acceptor molecules is ascribed primarily to the dispersion force and also to the electrostatic force. In contrast, the contribution of CT interactions to the attractive forces is minimal. We demonstrate that the highly directional feature of the exchange repulsion force, coupled with the attractive dispersion and electrostatic forces, is crucial in determining the intermolecular arrangements of actual CT crystals. These findings are key for understanding the unique structural and electronic properties of π-conjugated CT complexes.