China is a mountainous country, and Southwest mountain areas cover the most mountain areas in China and have the most serious problems. Taking Zhaotong city as the study area, based on 902 rural household questionnaires of 11 villages in 2 counties and Tobit model, this paper analyzes the geographical differences and influencing factors of energy consumption for non-production purposes of rural households living in different terrain conditions. This research finds that: (1) Coal takes up the main part of energy consumption in valley areas and coal consumption is mainly affected by per capita cultivated land area, household income, proportion of rural household energy expenditure in total expenditure, coal price, and family population size. Firewood takes up the main part of energy consumption in high mountain areas and firewood consumption is mainly affected by per capita firewood forest area, distance to purchase coal, household income, electricity price, and coal price. (2) Only when the distance is greater than 20 kilometers, that is the average distance of rural households living in middle mountain areas (1,600m∼1,800m) to purchase coal, the transportation condition has a significant impact on coal consumption. (3) In high mountain areas, prices of coal and electricity are the main factors influencing energy consumption choice of rural households. Too high prices of coal and electricity would to some extent lead rural households to choose firewood as the main energy consumption type. Compared to coal, rural households prefer to choose electricity.