Humans could exhibit distinct overall thermal preferences when exposed to identical indoor thermal environments, leading to preference groups such as "warm-likers" or "cool-likers", who consistently prefer warmer or cooler conditions than the average population, respectively. Currently, most thermal comfort modelling studies focus on capturing the instantaneous comfort states/preferences of occupants, while overlooking their overall thermal preferences. This paper proposes a benchmarking framework to identify and characterize overall thermal preferences based on the relationship between preferred setpoints and outdoor air temperatures, derived from the ECOBEE Donate Your Data program. Using descriptive statistics, we establish three temporally consistent overall preference groups, including warm-liker, moderate and cool-liker, along with a temporally chaotic preference group termed random. Our results demonstrate that warm-likers prefer temperature setpoints above 21.5 °C on heating days and 24–25 °C on cooling days, while cool-likers prefer setpoints below 19.6 °C on heating days and 22 °C on cooling days, indicating a significant impact of the seasonal transitions. We observed that around 50 % of users exhibit secondary overall preferences, implying that overall thermal preference could change over time. On average, overall thermal preference can be established in 10–16 setpoint adjustments. The study reveals varied responses to outdoor temperature changes among users: many maintain constant indoor temperature preferences, while a significant number adjust their indoor temperatures upwards by 0.1 °C–0.4 °C for each 1 °C rise in outdoor temperature. A smaller group prefers cooler indoor temperatures as it gets warmer outside, showing a unique negative adjustment trend of −0.1. We also found that climate interacts with the overall preference group, with warmer climates having more warm-likers and vice versa.