We report the development of Raman spectroscopy as a powerful tool for quantitative analysis of point defect and defect clusters in irradiated graphite. Highly oriented pyrolytic graphite (HOPG) was irradiated by 25 keV He+ and 20 keV D+ ions. Raman spectroscopy and transmission electron microscopy revealed a transformation of irradiated graphite into amorphous state. Annealing experiment indicated a close relation between Raman intensity ratio and vacancy concentration. The change of Raman spectra under irradiation was empirically analyzed by “disordered-region model,” which assumes the transformation from vacancy-contained region to disordered region. The model well explains the change of Raman spectra and predicts the critical dose of amorphization, but the nature of the disordered region is unclear. Then, we advanced the model into “dislocation accumulation model,” assigning the disordered region to dislocation dipole. Dislocation accumulation model can simulate the irradiation time dependencies of Raman intensity ratio and the c-axis expansion under irradiation, giving a relation between the absolute concentration of vacancy and Raman intensity ratio, suggesting an existence of the barrier on the mutual annihilation of vacancy and interstitial.