PurposeThis study aims to use deep neural networks (DNNs) to learn the conduction heat transfer physics and estimate temperature distribution images in a physical domain without using any physical model or mathematical governing equation.Design/methodology/approachTwo novel DNNs capable of learning the conduction heat transfer physics were defined. The first DNN (U-Net autoencoder residual network [UARN]) was designed to extract local and global features simultaneously. In the second DNN, a conditional generative adversarial network (CGAN) was used to enhance the accuracy of UARN, which is referred to as CGUARN. Then, novel loss functions, introduced based on outlier errors, were used to train the DNNs.FindingsA UARN neural network could learn the physics of heat transfer. Within a few epochs, it reached mean and outlier errors that other DNNs could never reach after many epochs. The composite outlier-mean error as a loss function showed excellent performance in training DNNs for physical images. A UARN could excellently capture local and global features of conduction heat transfer, whereas the composite error could accurately guide DNN to extract high-level information by estimating temperature distribution images.Originality/valueThis study offers a unique approach to estimating physical information, moving from traditional mathematical and physical models to machine learning approaches. Developing novel DNNs and loss functions has shown promising results, opening up new avenues in heat transfer physics and potentially other fields.