As temperature extremes become more frequent due to climate change, households need to maintain comfortable indoor environments and even reduce weather-related health issues. However, energy inequality exists as disadvantaged households are more constrained economically, hence they are more likely to limit their electricity consumption. This study empirically examines the responses to temperature among households of different wealth levels, using smart metering data for 2017–2019 in Changsha City, China. We introduce a metric termed Heating and Cooling Utilization Rate (HCUR), which quantifies the deviation of a household’s power consumption from its baseline level. We observe a U-shape relationship between HCUR and temperatures and also observe energy-limiting behaviors for lower-income households. The cooling-related consumption changes most during high-temperature nights. We find energy inequality is largest during 22–30 degrees Celsius when the least wealthy households are more likely to limit their consumption than the wealthiest ones. We also find heterogeneous responses to temperatures among households lived in residence with varying building characteristics. Projections under various climate change scenarios indicate that more cooling demand will be needed in the future, especially for less wealthy households. These findings highlight the importance of tailored interventions considering the heterogeneous energy consumption patterns among households of different wealth.