The sinuousness of the projected fringe patterns has a great influence on the accuracy of structured light-based 3D measurement. In this paper, a defocusing rectified multi-frequency patterns (DRMP) method is presented to eliminate the effect caused by the projector. The cause of phase error is derived and analyzed to prove the non-necessity of gamma correction in the DRMP method. First, the multi-frequency binary codes are projected onto the measured object, and a Gaussian smoothing filter is adopted to defocus the distorted fringe images captured by the imaging system. Then, an optimized phase unwrapping path which has the smallest phase error is proposed to obtain the unwrapped phase. After that, the stereo rectification and the two-step matching methods are applied to obtain the corresponding matching lines and points, respectively. Finally, we demonstrate that the proposed method can be implemented into the high-precision 3D shape measurement system. The validation experiments show that the proposed method is more precise and efficient compared with previous methods.