Owing to high strength to weight ratio, durability, corrosion resistance and design flexibility, composite materials are extensively used in engineering applications. While these materials have several useful properties, they often exhibit complex failure modes that arise from their heterogeneous microstructure details. A continuum model for composites should accordingly exploit the microstructural information for an accurate prediction of the non-local and non-linear behavior preceding failure. In simulating discontinuities such as cracks, a derivative-free continuum theory – peridynamics, to wit – has an advantage in that the integro-differential balance laws work even with discontinuities. It does not need the special measures required with continuum models using partial differential equations (PDEs) for simulating cracks. In this work, we use a novel non-local variant of the deformation gradient and propose a constitutive model for composite materials within a derivative-free set-up whilst incorporating the microstructural information, which enables a faithful reproduction of the macroscopic response leading to failure. Since concrete is perhaps the most commonly used composite material in large scale engineering applications, we implement the model to study fracture/damage and size effect in concrete. Our results are in close conformity with the experimental data available in the literature. We also show that complex phenomena like crack propagation and branching are accurately simulated via the proposed model with considerably reduced computational overhead.