Simple physical models for restricted diffusion in a potential, which provide important insights for NMR spin relaxation, usually are based on free diffusion within rigid boundaries or diffusion in relatively simple continuous potential energy surfaces. The diffusion-in-a-cone model is an example of the former and diffusion in an N-fold cosine potential is an example of the latter. The present work models restricted diffusion for arbitrary potential energy functions on the surface of a cone or a sphere, by expanding the potentials in Fourier or spherical harmonic series, respectively. The results exhibit simple relationships between generalized order parameters and effective correlation times, critical for analysis of experimental spin relaxation data, and illustrate the transition from diffusive-like to jump-like behavior in multi-well potentials.