The capabilities of image probe experiments are rapidly expanding, providing new information about quantum materials on unprecedented length and time scales. Many such materials feature inhomogeneous electronic properties with intricate pattern formation on the observable surface. This rich spatial structure contains information about interactions, dimensionality, and disorder -- a spatial encoding of the Hamiltonian driving the pattern formation. Image recognition techniques from machine learning are an excellent tool for interpreting information encoded in the spatial relationships in such images. Here, we develop a deep learning framework for using the rich information available in these spatial correlations in order to discover the underlying Hamiltonian driving the patterns. We first vet the method on a known case, scanning near-field optical microscopy on a thin film of VO2. We then apply our trained convolutional neural network architecture to new optical microscope images of a different VO2 film as it goes through the metal-insulator transition. We find that a two-dimensional Hamiltonian with both interactions and random field disorder is required to explain the intricate, fractal intertwining of metal and insulator domains during the transition. This detailed knowledge about the underlying Hamiltonian paves the way to using the model to control the pattern formation via, e.g., tailored hysteresis protocols. We also introduce a distribution-based confidence measure on the results of a multi-label classifier, which does not rely on adversarial training. In addition, we propose a new machine learning based criterion for diagnosing a physical system's proximity to criticality.