A phase field model is developed in this paper to study the influences of oxygen vacancies on domain evolutions under electric field in ${\text{BaTiO}}_{3}$ single crystal. The oxygen vacancies are modeled as defect dipoles, enabling the capture of domain memory effect. Three different domain structures are studied, and the distributions of defect density and the electromechanical responses under electric loading are simulated. The simulations reproduce well the reversible domain switching and large recoverable electric-field-induced strain observed in experiments on both engineered domain structure and single domain structure. It is also discovered that double hysteresis loop, stepwise hysteresis loop and shifted hysteresis loop can emerge due to vacancy induced depolarization field, which explains well the previous experimental observations. The results also indicate that domain memory effect and non-$180\ifmmode^\circ\else\textdegree\fi{}$ domain switching are the mechanisms for large recoverable electric-field-induced strain in ferroelectrics with oxygen vacancies.