A novel finite element model is proposed here for predicting the fracture toughness using real microstructural images and accounting for several parameters that can affect the crack propagation such as filler content, particle shape, particle agglomeration and particle debonding. The damage energy prior to the catastrophic failure of the whole microstructure is taken as the energy required for crack initiation, and the fracture toughness is calculated using the concept of a critical crack size. The predictions agree well with the measured values of the critical energy release rate at 20 °C as a function of both volume fraction and mean particle size. In addition, a parametric study showed that an increase in interfacial cohesive energy leads to higher fracture energies at 60 °C. The proposed methodology shows great potential and can be widely applied to other particulate composites, enabling industry to cost-effectively develop tougher, hence safer and more durable, particulate composites.