Objective. Transcranial magnetic stimulation (TMS) is a non-invasive brain stimulation method that is used to study brain function and conduct neuropsychiatric therapy. Computational methods that are commonly used for electric field (E-field) dosimetry of TMS are limited in accuracy and precision because of possible geometric errors introduced in the generation of head models by segmenting medical images into tissue types. This paper studies E-field prediction fidelity as a function of segmentation accuracy. Approach.The errors in the segmentation of medical images into tissue types are modeled as geometric uncertainty in the shape of the boundary between tissue types. For each tissue boundary realization, we then use an in-house boundary element method to perform a forward propagation analysis and quantify the impact of tissue boundary uncertainties on the induced cortical E-field. Main results. Our results indicate that predictions of E-field induced in the brain are negligibly sensitive to segmentation errors in scalp, skull and white matter (WM), compartments. In contrast, E-field predictions are highly sensitive to possible cerebrospinal fluid (CSF) segmentation errors. Specifically, the segmentation errors on the CSF and gray matter interface lead to higher E-field uncertainties in the gyral crowns, and the segmentation errors on CSF and WM interface lead to higher uncertainties in the sulci. Furthermore, the uncertainty of the average cortical E-fields over a region exhibits lower uncertainty relative to point-wise estimates. Significance. The accuracy of current cortical E-field simulations is limited by the accuracy of CSF segmentation accuracy. Other quantities of interest like the average of the E-field over a cortical region could provide a dose quantity that is robust to possible segmentation errors.