Phase retrieval composed of phase extracting and unwrapping is of great significance in different occasions, such as fringe projection based profilometry, quantitative interferometric microscopy and moire detections. Compared to phase extracting, phase unwrapping occupies most time consuming in phase retrieval, and it becomes an obstacle to realize real time measurements. In order to increase the calculation efficiency of phase retrieval as well as simplify its procedures, here, a high speed moire based phase retrieval method is proposed which is capable of calculating quantitative phase distributions without phase unwrapping or aberration compensation. We demonstrate the capability of the presented phase retrieval method by both theoretical analysis and experiments. It is believed that the proposed method will be useful in real time phase observations and measurements.