One of the basic criteria for the representativeness of composite material samples is the similarity of the quantitative and qualitative composition of the material in the sample to that of the material it represents. The article presents the results of experimental research concerning the impact of the ratio of the minimum size of cubic concrete samples (lmin) to the maximum size of aggregate grain (Dmax). The content of the basic components of concrete, i.e., coarse aggregate, mortar, sand and cement paste, was analysed. The content of these components in the concrete sample was identified by means of analysis of the sample cross-sectional image, and then, compared with the contents resulting from the concrete mixture proportion. Cases where lmin/Dmax was between 2.38 and 12.5 were analysed. The results of the study indicate a moderate correlation between these two parameters. The largest discrepancies between the component contents of the sample and the controlled concrete were usually recorded at the extreme values of the lmin/Dmax ratio. These discrepancies, depending on the component in question, range from approx. −15 to approx. +15%. Acceptable deviations (≤ ±5%) result in the value of this ratio within the estimated range between approximately 5.0 and approximately 10.0. In existing standards, the use of samples that meet the requirement lmin/Dmax ≥ 3.0 or 3.5 is recommended. In light of the presented test results, this requirement is sufficient for the material properties that are not sensitive to slight variation in concrete composition, i.e., compressive and tensile strength. The presented results are useful in the case of sample size selection for the study of concrete properties which are most sensitive to concrete composition changes. These properties include water absorption, permeability to liquids and gases, thermal properties, etc.