At a micro-scale level, grouted porous asphalt mixtures are multiphase composites consisting of aggregates, asphalt mortar, filler, asphalt mortar-aggregate interfacial transition zone, and asphalt mortar-filler interfacial transition zone. It exhibits different mechanical behavior under different temperatures and loading conditions. Compared with aggregate, asphalt mortar, and filler, the interfacial transition zone (ITZ) is more prone to damage at low temperatures, with both the fine structure and the nature of the contact interface having a significant effect on the overall strength and crack resistance of the material. To evaluate the influence of ITZ and filler material properties on the cracking resistance of the material, a grouted porous asphalt mixture model based on the real aggregate shape was established using finite element modeling. A cohesive zone model (CZM) was employed to characterize the internal mechanical behavior of the material, and a mesoscale damage simulation was performed for the grouted asphalt mixture. The study revealed the influence of different interfacial and filler material properties on the low temperature cracking resistance and proposed a design approach for anti-cracking grouted asphalt mixtures. The findings indicated that interfacial transition zone (ITZ) and filler properties play a crucial role on determining the damage characteristics. Increasing the ITZ and filler properties significantly enhanced the low temperature crack resistance of the grouted asphalt mixture.