The antenna phase calibration is critical for the millimeter wave (mmWave) hybrid beamforming communication system. Especially, with the increasing number of antennas being deployed in future massive MIMO systems, it is unrealistic to calibrate each antenna path in a microwave anechoic chamber. This work proposes a phase calibration based on over-the-air (OTA) scheme for mmWave hybrid beamforming systems, where the calibration process is performed automatically by the online signal transmission. The challenge for the OTA-based phase calibration is the imperfect channel estimation due to the phase error. To solve the challenge, we simplify the channel estimation process by integrating the beam alignment and phase calibration process. Consequently, the phase calibration is formatted to an MMSE-criterion based optimization problem solved by the Monte Carlo Markov Chain (MCMC) with probabilistic conversion improved Gibbs sampling method. Furthermore, an alternating iteration algorithm is developed to calibrate phase deviation for both the transmitter and receiver. We also extend our method to solve the joint amplitude and phase calibration problem. According to the simulation result, the calibration results outperform existing methods when the signal-to-noise ratio is less than 15dB. Besides, the proposed method exhibits a high beamforming gain during the calibration process, which paves the way for future massive device deployment in a practical system.