Circular and spherical particle models are a class of discrete elements (DEM) that have been increasingly applied to fracture studies of quasi-brittle materials, such as rock and concrete, due to their proven ability to simulate fracture processes through random particle assemblies representing quasi-brittle materials at the grain scale. More recently, DEM models have been applied to old stone masonry fracture studies. In order to extend its applicability to structures of larger dimensions, an enhanced hybrid particle model is proposed here that allows finite elements with a given surface roughness, provided by the discretization of the element boundary with particles, to interact with the particulate media in which they are embedded. The performance of the hybrid model is compared with that of a traditional all-particle model under uniaxial testing. It is shown that similar results are obtained, namely, in the elastic phase, figures of rupture and pre-peak and post-peak behavior, while the hybrid model allows for a significant computational run time reduction of 20% to 25% in the coarse particle assemblies. Finally, the proposed hybrid model is applied in the simulation of shear tests of stone masonry walls and dry and mortared joints, providing reasonably good agreement with both the experimental results and predictions. For the rubble masonry tests, the hybrid model allows for a computation run time reduction of around 40% when compared with an all-particle model.