Nonlinear optical wave mixing is a widely used method to produce light with new frequencies that has a significant impact on laser technology and optical imaging. The most important figure of merit in wave-mixing processes, i.e., high conversion efficiency, is always required in laser applications. We demonstrate a method to recover high conversion efficiency of second harmonic generation in a BaMgF4 single crystal with weakly scattering defects via feedback-based wavefront shaping under birefringent phase-matching condition. By optimizing the fundamental wavefront, a typical second harmonic output with an enhancement factor of 1.14 and a corresponding recovery efficiency of 86.3% is displayed. This investigation may modify the wide understanding of scattering in crystals and provide an avenue to recover the nonlinear optical conversion efficiency in crystals with various defects without special fabrications.