The paper aims at developing a simple two-step homogenization scheme for prediction of elastic properties of a high performance concrete (HPC) in which microstructural heterogeneities are distinguished with the help of nanoindentation. The main components of the analyzed material include blended cement, fly-ash and fine aggregate. The material heterogeneity appears on several length scales as well as porosity that is accounted for in the model. Grid nanoindentation is applied as a fundamental source of elastic properties of individual microstructural phases in which subsequent statistical evaluation and deconvolution of phase properties are employed. The multilevel porosity is evaluated from combined sources, namely mercury intrusion porosimetry and optical image analyses. Micromechanical data serve as input parameters for analytical (Mori–Tanaka) and numerical FFT-based elastic homogenizations at microscale. Both schemes give similar results and justify the isotropic character of the material. The elastic stiffness matrices are derived from individual phase properties and directly from the grid nanoindentation data with very good agreement. The second material level, which accounts for large air porosity and aggregate, is treated with analytical homogenization to predict the overall composite properties. The results are compared with macroscopic experimental measurements received from static and dynamic tests. Also here, good agreement was achieved within the experimental error, which includes microscale phase interactions in a very dense heterogeneous composite matrix. The methodology applied in this paper gives promising results for the better prediction of HPC elastic properties and for further reduction of expensive experimental works that must be, otherwise, performed on macroscopic level.