High-quality measurement of temperature distribution information is of great significance for industrial production. As a typical non-contact temperature measurement method, acoustic tomography (AT) can obtain temperature distribution through reconstruction algorithms. To improve the reconstruction quality, a two-stage algorithm based on virtual observation (VO) and residual network (ResNet) was proposed. This algorithm combines the advantages of VO and ResNet. Aiming at total least squares, the VO method was adopted to reconstruct the ultrasonic time-of-flight (TOF) to obtain the temperature distribution under a coarse grid. Then, ResNet was built to predict the temperature distribution under the refined grid, and the dual-input model was used to improve the network generalization ability. At the same time, the sub-pixel convolution layer was introduced to reduce the network computing dimension, which improves the computing efficiency and the reconstruction quality. The numerical simulation of typical temperature field models was carried out, and the reconstruction results show that the average relative error and root mean square error are less than 0.86% and 1.21%, respectively. The noise immunity and reconstruction spatial resolution are better than traditional algorithms and the previous two-stage algorithm.