Fringe projection is an important technology in three-dimensional measurement and target recognition. The measurement accuracy depends heavily on the calibration of the absolute phase and projector pixels. An easy-to-implement calibration method based on the Random Sample Consensus (RANSAC) algorithm is proposed to exterminate the phase error data and elevate the measurement accuracy in a fringe projection system. The reconstruction experiments of a double-sphere standard demonstrate that the uncertainties in radius and sphere-distance measurement are reduced to one thousandth of the measured value or even less, and the standard deviation in multiple measurements is restricted to within 50 μm. The measurement accuracy provided by the proposed RANSAC method can be improved by up to 44% compared with that provided by traditional least squared method (LSM). The proposed calibration method is easy and simple to implement, and it does not need additional hardware, but rather a calibration board.