Objectives Evolutionary changes in hominin social complexity have been associated with increases in absolute brain size. The temporal lobes are nestled in the middle cranial fossae (MCF) of the skull, the dimensions of which allow estimation of temporal lobe volume (TLV) in extant and fossil taxa. Materials and Methods The main aim of this study is to determine where along the hominid phylogeny, major temporal lobe size transitions occurred. We used computed tomography (CT) scans of crania, 3D photogrammetry data, and laser surface scans of endocranial casts to measure seven MCF metrics in 11 extant anthropoid taxa using multiple regressions to estimate TLV in 5 extant hominids and 10 fossil hominins. Phylogenetic comparative methods mapped temporal lobe size, brain size, and temporal lobe proportions onto phylogenetic trees broadly for Hominidae and specifically for Hominini. Results Extant Homo sapiens were not an outlier in relative brain size, temporal lobe size, or proportions of the temporal lobes, but some proportions within the lobe were uniquely altered. The most notable changes in relative temporal lobe size and proportions saw a decrease in relative temporal lobe size and proportions in the genus Pan compared to other extant great apes and fossil hominins while there was a relative increase in the temporal lobe width and length in Australopithecus-Paranthropus clade compared to the genus Homo and other extant great apes including modern humans. Discussion We do not find support for the social brain, environmental or functional craniology hypotheses alone but think it prudent to consider the implications of cerebral reorganization between the temporal lobes and other regions of the brain within the context of these hypotheses and with future investigation is warranted.