Understanding the controlling mechanisms of γ columnar grain size in the weld metal of low-carbon, low-alloy steel is critical for optimizing resultant microstructures and the weld metal’s ensuing properties. Here, we investigate the role of welding flux composition upon the γ columnar grain size by submerged arc welding of EH36 shipbuilding steel with designed CaF2-SiO2 fluxes. We found that the addition of SiO2 from 5 to 40 mass-% increases average columnar grain size by a factor of nearly 2.5, which results from the change of the weld pool solidification mode from peritectic to primary δ solidification. Such a change is directly related to element transfer behaviors between the flux and the weld metal. Furthermore, we offer compelling evidence that alterations in γ columnar grain size are not primarily governed by significantly populated inclusions and/or weld metal macro-morphological changes. Our findings may largely serve as a viable strategy toward designing welding consumables to match base metals to ensure the soundness of the weldment.