With X-ray radiation protection and dose management constantly gaining interest in interventional radiology, novel procedures often undergo prospective dose studies using anthropomorphic phantoms to determine expected reference organ-equivalent dose values. Due to inherent uncertainties, such as impact of exact patient positioning, generalized geometry of the phantoms, limited dosimeter positioning options, and composition of tissue-equivalent materials, these dose values might not allow for patient-specific risk assessment. Therefore, first the aim of this study is to quantify the influence of these parameters on local X-ray dose to evaluate their relevance in the assessment of patient-specific organ doses. Second, this knowledge further enables validating a simulation approach, which allows employing physiological material models and patient-specific geometries. Phantom dosimetry experiments using MOSFET dosimeters were conducted reproducing imaging scenarios in prostatic arterial embolization (PAE). Associated organ-equivalent dose of prostate, bladder, colon, and skin was determined. Dose deviation induced by possible small displacements of the patient was reproduced by moving the X-ray source. Dose deviation induced by geometric and material differences was investigated by analyzing two different commonly used phantoms. We reconstructed the experiments using Monte Carlo (MC) simulations, a reference male geometry, and different material properties to validate simulations and experiments against each other. Overall, MC-simulated organ dose values are in accordance with the measured ones for the majority of cases. Marginal displacements of X-ray source relative to the phantoms lead to deviations of 6-135% in organ dose values, while skin dose remains relatively constant. Regarding the impact of phantom material composition, underestimation of internal organ dose values by 12-20% is prevalent in all simulated phantoms. Skin dose, however, can be estimated with low deviation of 1-8% at least for two materials. Prospective reference dose studies might not extend to precise patient-specific dose assessment. Therefore, online organ dose assessment tools, based on advanced patient modeling and MC methods, are desirable.