We investigate in this paper analytically and numerically by means of 3D simulations the viscoelastic behavior of concrete and mortar subjected to creep loading and moderate temperatures at mesoscale. These heterogeneous materials are assumed to be composed of thermoelastic aggregates distributed in a linear thermoviscoelastic matrix; moreover, the Interfacial Transition Zones (ITZ) between aggregates and matrix, whose behavior is also considered as linear thermoviscoelastic, are explicitly introduced. The numerical specimens consist in unstructured periodic meshes containing polyhedral aggregates with various size and shapes randomly distributed in a box. Zero-thickness interface finite elements are introduced between aggregates and matrix to model the ITZ. Macroscopic response and averaged stresses and strains in the matrix and aggregate phases are compared to analytical estimations obtained with classical mean-field approximation schemes applied in the Laplace–Carson space, in which the ITZ are introduced via imperfect interfaces modeled with the Linear Spring Model (LSM). The effects of ITZ thickness, aggregate shape and uniform temperature increase are then studied to evaluate their respective influence on the local and macroscopic creep behavior of mortar and concrete. Globally, it is found that the model response is in relatively good agreement with numerical simulations results, and that as expected while the ITZ do not affect significantly the concrete behavior, they have a non-negligible impact on the mortar one.