The fine microstructure resulting from the martensitic transformation drives many of the desired mechanical properties for quench-and-temper steels. The scale of martensite features correlates with austenite grain size, such that finer austenite grains produce finer martensitic structures. Measuring prior austenite grain (PAG) size from observations of the hierarchical martensitic structure using etch-based techniques remains challenging, especially for fine-grained specimens with low C and P content. This has driven an interest in our field in reconstructing prior austenite microstructure from electron backscatter diffraction (EBSD) data collected on tempered martensite. However, reliably reconstructing the data is not straightforward because (1) up to 24 variants can theoretically be observed from a single PAG; (2) the shear-induced plasticity that arises from martensite formation produces significant scatter in the pole figure; (3) there is inherent noise in Hough-indexed EBSD data; (4) annealing twins exist within many PAGs, producing martensite variants of similar orientations when compared with the parent grains; and (5) variation in austenite orientation can occur across a single PAG. To the author's knowledge, the majority of reconstruction algorithms published to date utilize either point-to-point or flood-fill approaches, which can produce artifacts due to the relatively high probability of adjacent EBSD observations falling within any reasonable tolerance angle with 24 variants in a parent and four annealing twin orientations. Therefore, we propose a probabilistic austenite reconstruction technique based on a clustering algorithm known as graph cutting. We demonstrate its application to steels and a binary ferrous alloy, then validate the results against chemical etching and retained austenite orientations. The graph cutting technique utilizes the misorientation distribution of martensitic variants and the potential austenite orientations for each observation, as estimated via Bayesian inference of the experimental orientation relationship and its scatter. It is found that this technique produces accurate reconstructions with identification of annealing twins in the PAGs, even in the presence of a large fraction of poorly indexed data points.