The acoustic temperature measurement method has a broad application prospect due to its advantages of high precision, non-contact, etc. It is expected to become a new method for hidden fire source detection in mines. The acoustic time of flight (TOF) can directly affect the accuracy of acoustic temperature measurement. We proposed a quadratic correlation-based phase transform weighting (PHAT-β) algorithm for estimating the time delay of the acoustic temperature measurement of a loose coal. Validation was performed using an independently built experimental system for acoustic temperature measurement of loose coals under multi-factor coupling. The results show that the PHAT-β algorithm estimated acoustic TOF values closest to the reference line as the sound travelling distance increased. The results of coal temperature inversion experiments show that the absolute error of the PHAT-β algorithm never exceeds 1 °C, with a maximum value of 0.862 °C. Using the ROTH weighted error maximum, when the particle of the coal samples is 3.0–5.0 cm, the absolute error maximum is 4.896 °C, which is a difference of 3.693 °C from the error minimum of 1.203 °C in this particle size interval. The accuracy of six algorithms was ranked as PHAT-β > GCC > PHAT > SCOT > HB > ROTH, further validating the accuracy and reliability of the PHAT-β algorithm.