Micro-Raman spectroscopy is applied to evaluate change of the crystal quality in molecular beam epitaxial GaAs layers on Si after rapid thermal annealing (RTA). The forbidden transverse optical phonon is observed in the GaAs layers especially near the interface. In the as-grown state, the Raman frequency of the longitudinal optical phonon shifts toward higher frequency near the interface. This blue shift indicates the existence of the compressive stress due to the lattice mismatch between GaAs and Si. On the other hand, after the RTA, the Raman peak shifts toward lower frequency. This red shift indicates that the tensile stress exists near the interface because of the difference in thermal expansion. The stress change indicates the relaxation of the lattice mismatch stress near the interface by formation of dislocations during the RTA.