The lack of thermal comfort for occupants of a living space owing to economic and environmental impacts is a serious problem that reveals the disconnection between energy efficiency and thermal comfort. Many studies have focused on human physical responses to predict personal thermal comfort and estimated thermal sensation using the skin temperature of local points throughout the face or body shown in infrared images. However, the prediction accuracy was inadequate because the statistical significance of skin temperatures extracted from several designated local points was insufficient to represent the thermal sensation of the entire body. Thus, in this study, a thermal image was preprocessed to represent a clear body thermal distribution and input into a deep convolutional neural network classifier to predict the personal thermal sensation vote (TSV). Furthermore, the feature map was numerically analyzed to understand the human thermal interaction with clothing rate and environmental factors. Consequently, an accuracy and F1-score of 96% were achieved. Moreover, the results revealed that the local features and relationship between adjacent temperatures shown in infrared images could represent the thermal sensation of the entire body. In addition, the feature map generated from the developed model exhibited numerically significant differences according to the variations in TSV. The high accuracy of the proposed method implies that it can represent most of the information used in previous studies that considered the local skin temperature extracted from thermal images.