Microstructures are critical to the physical properties of materials. Stochastic microstructures are commonly observed in many kinds of materials (e.g., composite polymers, multiphase alloys, ceramics, etc.) and traditional descriptor-based image analysis of them can be challenging. In this paper, we introduce a powerful and versatile score-based framework for analyzing nonstationarity in stochastic materials microstructures. The framework involves training a parametric supervised learning model to predict a pixel value using neighboring pixels in images of microstructures (as known as micrographs), and this predictive model provides an implicit characterization of the stochastic nature of the microstructure. The basis for our approach is the Fisher score vector, defined as the gradient of the log-likelihood with respect to the parameters of the predictive model, at each micrograph pixel. A fundamental property of the score vector is that it is zero-mean if the predictive relationship in the vicinity of that pixel remains unchanged, which we equate with the local stochastic nature of the microstructure remaining unchanged. Conversely, if the local stochastic nature changes, then the mean of the score vector generally differs from zero. In light of this, our framework analyzes how the local mean of the score vector varies across one or more image samples to: (1) monitor for nonstationarity by indicating whether new samples are statistically different than reference samples and where they may differ and (2) diagnose nonstationarity by identifying the distinct types of stochastic microstructures that are present over a set of samples and labeling accordingly the corresponding regions of the samples. Unlike feature-based methods, our approach is almost completely general and requires no prior knowledge of the nature of the nonstationarities or the microstructure itself. Using a number of real and simulated micrographs, including polymer composites and multiphase alloys, we demonstrate the power and versatility of the approach.