Models linking surface characteristics within incident solar radiation are inexorably dependent on the topography of the given region. To date, however, most operational surface reflectance retrievals treat this dependence by assuming a flat terrain, leading to significant deviations in the estimated reflectance. Here, we demonstrate that incorporating dynamic topography directly into the joint surface and atmospheric model during retrievals has several advantages. First, it allows for a more complete physical accounting of downwelling illumination, providing more accurate estimates of the absolute magnitude of reflectance. Second, it facilitates a superior resolution of the atmospheric state, most notably due to the confounding influence of atmospheric aerosols and unresolved topographic effects. Our methodology utilizes a practical, high-fidelity, model-driven approach to separate out diffuse and direct irradiation and account for topographic effects during the joint inversion of atmosphere and surface properties. We achieve this by enhancing the atmosphere/surface inversion to account for the radiative transfer effects of surface slope. We further demonstrate how uncertainties in topographic features can be quantified and leveraged within our formulation for a more realistic posterior uncertainty estimates. Our results demonstrate that the inclusion of topographic effects into the retrieval model reduces errors in the reflectance of an only moderately rugged terrain by more than 15%, and that a post hoc accounting of topography cannot achieve these same results.