The ultrafast surface plasmon resonance nonlinearities and their connection with the conduction band electron dynamics are discussed in metal nanoparticles in the light of the results of high sensitivity femtosecond pump-probe experiments in silver nanoparticles embedded in a glass matrix. The optical response is interpreted in terms of frequency shift and broadening of the surface plasmon resonance and is related to the changes of the metal nanoparticle dielectric function induced by ultrafast perturbation of the electron distribution. Alteration of the interband absorption is found to be responsible for the observed red shift and very short time delay broadening of the surface plasmon resonance, in agreement with numerical simulations and with measurements in silver films. On a longer time scale, a new nonlinear mechanism due to increase of the electron scattering off the surfaces is demonstrated. This mechanism, specific to confined system, plays an important role in the ultrafast nonlinear optical response of small nanoparticles.