Residents themselves are responsible for controlling their living environment, and their perception of coldness is important to protect their health. Although previous studies examined the association between perceived coldness and indoor temperature, they did not consider the spatial–temporal differences in indoor temperatures. This study, conducted in Japan, measured indoor temperatures in 1,553 houses across several rooms (living room, changing room, and bedroom) and heights (at 1 m above the floor and near the floor) over two weeks and obtained the perceived coldness from 2,793 participants during winter. Results showed substantial temperature differences between rooms (horizontal differences): 3.8 °C between living and changing rooms, and 4.1 °C between living rooms and bedrooms. The average vertical and diel (evening-morning) temperature differences in the living room were 3.1 °C and 3.0 °C, respectively. Regional analysis revealed that the Tohoku region experienced larger horizontal and diel indoor temperature differences, primarily due to its practice of intermittent and partial heating in living rooms only, in contrast to Hokkaido's approach of heating the entire house continuously. Despite Hokkaido's comprehensive heating system, it exhibited the largest vertical temperature difference of 5.1 °C in living rooms, highlighting the insufficiency of heating alone and the necessity for enhanced thermal insulation. The multivariate logistic regression analyses showed that average temperatures and vertical temperature differences were associated with perceived coldness, while horizontal and diel differences did not show a significant association, further emphasizing the importance of improved thermal insulation. Moreover, factors like individual attributes (age and gender), and lifestyle choices (meal quantity, exercise habits, alcohol consumption, and clothing amount) were significantly associated with coldness perception. Notably, older adults were less likely to perceive coldness but more vulnerable to the health impacts of low temperatures, underscoring the necessity of not solely relying on human perception for indoor temperature management to protect cold-related health problems.