Experiments have been carried out to ‘freeze’ the friction stir welding process by stopping the tool and immediately quenching the work piece in an Al-2195 plate welded under typical conditions. Sectioning through the ‘frozen’ weld keyhole with the tool in place has allowed the microstructure development, leading to the formation of the ultrafine grained nugget material, to be directly observed as fresh material encounters the deformation field surrounding the rotating pin. The grain refinement process is shown to be driven by grain subdivision at the colder periphery of the tools deformation zone, and the geometric effects of strain, which together reduce the overall high angle boundary spacing with increasing deformation. However, it also involves thermally activated high angle grain boundary migration, which increases as the temperature rises towards the tool. The higher temperature latter stages of the refinement process are closely reminiscent of geometric dynamic recrystallisation seen in high strain hot torsion experiments. The nugget grain structure has also been found to become more equiaxed and coarsens slightly, due to static annealing in the thermal wake of the tool.