We perform inversions of gravity data (geopotential model EGM96) and seismic tomography model (S16RLBM) for the scaling factor ( ζ), which relates relative density anomalies to relative S-wave velocity anomalies. The gravity data and tomographic model are anti-correlated below continents down to a depth of z=200 km. This anti-correlation is not present below oceans. Except for smoothness, which is controlled by a damping factor, no a priori information is added to the inversion. Data are filtered between degrees ℓ=11 and ℓ=16 of the spherical harmonic expansion. This spectral window is well suited for the study of intermediate-size (2000–4000 km) anomalies in the uppermost mantle. Calculations are made separately for sub-continental and sub-oceanic mantle. The sub-continental and sub-oceanic scaling factors are significantly different at depths shallower than 260 km. In both cases, the magnitude of ζ is around 0.05. The sub-continental scaling factor has a positive root down to z=220 km, whereas the sub-oceanic scaling factor yields positive values down to z=140 km only. At depth shallower than 350 km, models of ζ do not depend on the damping factor or the viscosity model. At depths greater than 350 km, the resolution of ζ( z) decreases significantly and low degrees (ℓ=2–4) add information from large-scale anomalies and from the lower mantle. As a result, the shape and values of ζ for ℓ=2–16 and ℓ=11–16 are significantly different at depths greater than 350 km. A possible explanation of the discrepancies between the sub-continental and sub-oceanic scaling factor is that intermediate-scale anomalies are more important in the continental uppermost mantle than in the oceanic uppermost mantle.