This study aims to explore the influence of various aging temperatures on the microstructure and mechanical properties of TC21 forgings. Basket-weave microstructures were prepared via quasi-β forging followed by annealed at 900 °C for 1.5 h and aged at 530 °C, 560 °C, and 590 °C for 4 h, and then their tensile performance, impact toughness, and fracture toughness were examined. Results indicate that higher aging temperatures not only facilitate the segregation of Al element, resulting in a reduction in the effective size (dαlaths, the width/length ratio) of primary α laths (αlaths) and their amount, but also promote phase transition, inducing an increase in the effective size (dαfine) of secondary fine α (αfine) lamellae. While both dαlaths−1/2 and dαfine−1/2 have a linear relationship with the strength, which was analyzed using the Hall–Petch relationship, the more pronounced interface strengthening effect and hindering crack propagation mainly results from the αlaths. Moreover, the plasticity declines significantly as the proportion of the αlaths phases decreases. Thus, for strength and plasticity, the αlaths are a much more important factor than the αfine lamellae. However, although the synergistic deformation and fracture of both the αlaths and αfine phases can consume a larger impact initiation energy, the enhancement of the impact energy of the alloy is primarily related to a high precipitation amount of the αfine lamellae. Furthermore, the decrease in dαlaths promotes the misorientation and accumulation of dislocations between αlaths interfaces, hindering the enhancement of KIC (fracture toughness) by forming transgranular fractures. Conversely, the increase in KIC attributed to the energy consumed in the plastic deformation region at the crack tip, due to the αfine lamellae compensates for the energy, resulting in a reduction in the crack propagation path. Therefore, the αfine lamellae may play a more crucial role in increasing the toughness of alloys, in particular the impact toughness of the TC21 alloy.