In this paper, we study the local rigidity of constant mean curvature (CMC) hypersurfaces. Let x:Mn→Mn+1(c),n≥4, be a piece of immersed constant mean curvature hypersurface in the (n+1)-dimensional space form Mn+1(c). We prove that if the scalar curvature R is constant and the number g of the distinct principal curvatures satisfies g≤3, then Mn is an isoparametric hypersurface. Further, if Mn is a minimal hypersurface, then Mn is a totally geodesic hypersurface for c≤0, and Mn is either a Cartan minimal hypersurface, a Clifford minimal hypersurface, or a totally geodesic hypersurface for c>0, which solves the high dimensional version of Bryant Conjecture.