We describe the measurement and modeling of amplitude noise and phase noise in ultra-high Q nanomechanical resonators made from stoichiometric silicon nitride. With quality factors exceeding 2 million, the resonators' noise performance is studied with high precision. We find that the amplitude noise can be well described by the thermomechanical model, however, the resonators exhibit sizable extra phase noise due to their intrinsic frequency fluctuations. We develop a method to extract the resonator frequency fluctuation of a driven resonator and obtain a noise spectrum with dependence, which could be attributed to defect motion with broadly distributed relaxation times.