In reinforced concrete (RC) structures, the compressive strength of concrete can play a crucial role in seismic performance and is usually difficult to estimate. Major seismic codes prescribe that concrete strength must be determined essentially from in situ and laboratory tests. Mean values obtained from such tests are the reference design values when assessing existing structures under seismic actions. The variability of concrete strength can also play an important role, generally requiring that various homogeneous domains are identified in a single structure, in each of which a specific mean value should be assumed as representative. This study analyzes the inter- and intra-variability of the concrete strength of existing buildings using a very large database made up of approximately 1600 core tests extracted from RC buildings located in the Basilicata region (Southern Italy). The analysis highlighted that concrete strength variability was dependent on the structures’ dimensions as well as on the number of storeys. Moreover, the concrete strength of cores extracted from columns was found to be, on average, lower than that from beams, thus justifying the usual practice to extract cores mainly from columns, which results in a conservative approach as well as a more feasible one. Finally, some case studies were analyzed, specifically focusing on the effects of the within-storey variability. Conservative strength values, to be used especially in the case of vertical members subjected to high axial loads, are suggested.