The biological function of membranes is closely related to their softness, which is often studied through the membranes' thermally driven fluctuations. Typically, the analysis assumes that the relaxation rate of a pure bending deformation is determined by the competition between membrane bending rigidity and viscous dissipation in the surrounding medium. Here, we reexamine this assumption and demonstrate that viscous flows within the membrane dominate the dynamics of bending fluctuations of nonplanar membranes with a radius of curvature smaller than the Saffman-Delbrück length. Using flickering spectroscopy of giant vesicles made of dipalmitoylphosphatidylcholine, DPPC:cholesterol mixtures and pure diblock-copolymer membranes, we experimentally detect the signature of membrane dissipation in curvature fluctuations. We show that membrane viscosity can be reliably obtained from the short time behavior of the shape time correlations. The results indicate that the DPPC:cholesterol membranes behave as a Newtonian fluid, while the polymer membranes exhibit more complex rheology. Our study provides physical insights into the time scales of curvature remodeling of biological and synthetic membranes.