Imaging defects in austenitic welds presents a significant challenge for the ultrasonic non-destructive testing community. Due to the heating process during their manufacture, a dendritic structure develops, exhibiting large grains with locally anisotropic properties which cause the ultrasonic waves to scatter and refract. When basic imaging algorithms, which typically make constant wave speed assumptions, are applied to datasets arising from the inspection of these welds, the resulting defect reconstructions are often distorted and difficult to interpret correctly. However, knowledge of the underlying spatially varying material properties allows correction of the expected wave travel times and thus results in more reliable defect reconstructions. In this paper, an approximation to the underlying, locally anisotropic structure of the weld is constructed from ultrasonic time of flight data. A new forward model of wave front propagation in locally anisotropic media is presented and used within the reversible-jump Markov chain Monte Carlo method to invert for the map of effective grain orientations across different regions of the weld. This forward model and estimated map are then used as the basis for an advanced imaging algorithm and the resulting defect reconstructions exhibit a significant improvement across multiple flaw characterization metrics.