T cell receptors (TCRs) recognize peptides bound and presented by major histocompatibility complex (MHC) proteins using multiple complementarity determining region (CDR) loops. While numerous analyses have illuminated structural and biophysical aspects of TCR recognition, how the distribution of binding free energy within TCR-pMHC interfaces promotes unique TCR recognition features, including MHC restriction and the apparent dichotomy of specificity and cross-reactivity, remains unclear. Utilizing double mutant cycles, here we performed a comprehensive structural and thermodynamic deconstruction of the interaction between the A6 TCR and the Tax peptide presented by the class I MHC HLA-A2. In contrast with general expectations, we observed that the central regions of the peptide and its interactions with the hypervariable CDR3 loops contribute little to specificity, instead promoting by dynamic effects the cross-reactivity that is a hallmark of TCR recognition. We also observed that TCR restriction towards HLA-A2 results from not from conserved interactions with the germline loops, but instead from strong interactions with the hypervariable CDR3 loop of the α chain. Formation of these latter interactions, however, is dependent upon the unique structural properties of the peptide, highlighting that TCR specificity towards peptide and MHC can emerge from the need to engage a unique, composite peptide/MHC interface with tightly coupled structural properties.