In this work we aim to develop a unified mathematical framework and a reliable computational approach to model the brittle fracture in heterogeneous materials with variability in material microstructures, and to provide statistical metrics for quantities of interest, such as the fracture toughness. To describe the material responses such as the nucleation and growth of fractures in a uniform setting, we consider the peridynamics model. In particular, a stochastic state-based peridynamic model is developed, where the micromechanical parameters are modeled by a finite-dimensional random field, e.g., a combination of truncated random variables determined by the Karhunen–Loève decomposition or the principal component analysis (PCA). To solve this stochastic peridynamic problem, probabilistic collocation method (PCM) is employed to sample the random field that represents the micromechanical parameters. For each sample, the corresponding peridynamic problem is solved by an optimization-based meshfree quadrature rule. We present rigorous analysis for the proposed scheme and verify the convergence rate with a number of benchmark problems. The proposed scheme not only possesses the asymptotic compatibility spatially but also achieves an algebraic or sub-exponential convergence rate in the parametric random space when the number of collocation points grows. Finally, to validate the applicability of this approach on real-world fracture problems, we consider the problem of crystallization toughening in glass-ceramic materials, in which the material at the microstructural scale contains both amorphous glass and crystalline phases. The proposed stochastic peridynamic solver is employed to capture the crack initiation and its growth of the glass-ceramics with different crystal volume fractions, and the averaged fracture toughness are also calculated accordingly. The numerical estimates of fracture toughness show good consistency with data from experimental measurements.