Mantle viscosity exerts important controls on the long-term (i.e., >106 years) dynamics of the mantle and lithosphere and the short-term (i.e., 10 to 104 years) crustal motion induced by loading forces including ice melting, sea-level changes, and earthquakes. However, mantle viscosity structures inferred from modeling observations associated with mantle dynamic and loading processes may differ significantly and remain a hotly debated topic over recent decades. In this study, we investigate the effects of mantle viscosity structures on observations of the geoid, mantle structures, and present-day crustal motions and time-varying gravity by considering five representative mantle viscosity structures in models of mantle convection and glacial isostatic adjustment (GIA). These five viscosity models fall into two categories: 1) two viscosity models derived from modeling the geoid in mantle convection models with ∼100 times more viscous lower mantle than the upper mantle, and 2) the other three with less viscosity increase from the upper to lower mantles that are derived from modeling the late Pleistocene and Holocene relative sea level changes and other observations in GIA models. Our convection models use the plate motion history for the last 130 Myrs as the surface boundary conditions and depth- and temperature-dependent viscosity to predict the present-day convective mantle structure of subducted slabs and the intermediate wavelength (degrees 4–12) geoid. Our GIA models using different ice history models (e.g., ICE-6 G and ANU) compute the GIA-induced present-day crustal motions and time-varying gravity. Our calculations demonstrate that while the viscosity models with a higher viscosity in the lower mantle (∼2 × 1022 Pa.s) reproduce the degrees 4–12 geoid and seismic slab structures, they significantly over-predict the geodetic (i.e., GPS and GRACE) observations of crustal motions and time varying gravity. Our calculations also show that while two viscosity models derived from fitting the RSL data with averaged mantle viscosity of ∼1021 Pa.s for the top 1200 km of the mantle reproduce well the geodetic observations independent of ice models, they fail to explain the geoid and seismic slab structures. Therefore, our study highlights the persisting conundrum of mantle viscosity structures derived from different observations. We also discuss a number of possible ways including transient, stress-dependent and 3-D viscosity to resolve this important issue in Geodynamics.