Finite element models generating the random aggregate have been used to predict the dynamic moduli of the asphalt concrete materials. The asphalt concrete is modeled as a triphasic medium composed of granular inclusions with linear elastic properties, interfacial transition zone (ITZ) with linear elastic properties, and a porous matrix of bituminous mastics exhibiting linear viscoelastic behavior at small strain level. The predicted complex moduli of asphalt mixes from numerical approach are found close to that from the dynamic modulus tests, resulting in a good agreement. Meanwhile, the size effort of dynamic modulus for asphalt concrete material is investigated through nearly 30,000 times numerical examples, and a theoretical statistic formula for describing the size effort of dynamic modulus is proposed. The simulation results reveal that the theoretical statistic formula provides a suitable description for the size effort of dynamic modulus for asphalt concrete.