We investigate a recent controversy in ultrafast magnetization dynamics by comparing the demagnetization rates from two frequently used but competing descriptions for finite temperature magnetism, namely a rigid band structure Stoner-like approach and a system of localized spins. The calculations on the localized spin system show a demagnetization rate and time comparable to experimentally obtained values, whereas the rigid band approach yields negligible demagnetization, even when the microscopic spin-flip process is assumed to be instantaneous. This shows that rigid band structure calculations will never be in quantitative agreement with experiments, irrespective of the investigated microscopic scattering mechanism.