We propose an idea that the observed CP violation in neutrino oscillation is originated from the phase in right-handed neutrino mixing by seesaw mechanism. We add small breaking terms $M_{ij}N_{R_i}N_{R_j} $ in model based on $A_4$ symmetry to generate non-diagonal right-handed neutrino mixing, which will give a small correction to TBM mixing via seesaw mechanism with nonzero reactor angle and CP violating phase. We estimate the CP violating phase by investigating the process of leptogenesis due to the decay of right-handed neutrino.