Hierarchical multiscale modeling of heterogeneous materials has traditionally relied upon a deterministic estimation of constitutive properties when making microstructure-sensitive predictions of effective response at each subsequent length-scale. Such an approach is wholly unsuitable for a variety of material classes, such as ceramic matrix composites, which exhibit large variability at multiple length-scales. This work demonstrates a framework for approaching two open problems towards improved microstructure-sensitive predictions, namely, (i) probabilistically calibrating complex constitutive models at the mesoscale to sparsely observed macroscale experimental data, and (ii) propagating this stochastic constituent behavior at the mesoscale towards low-cost homogenized predictions for unseen microstructures. The proposed stochastic scale-bridging framework displays a continuity of information flow where no portion of the experimental data is neglected out of convenience, facilitating the greatest information gain from oftentimes costly experiments. In this paper, suitable protocols were developed to address the challenges described above. The protocols were subsequently demonstrated on ceramic matrix composite’s uniaxial tensile stress–strain response, where constituent behavior at the mesoscale was described using continuum damage mechanics, and predictions encapsulating constitutive model parameter uncertainty were made for novel microstructures. The methodology presented in this work is broadly applicable to various material classes and constitutive models with high-dimensional parameter sets.