Chemical composition in PM2.5 (particulate matter with a diameter less than 2.5 μm) is essential information for evaluating their climate, environment and health effects. Concentrations of carbonaceous components and water-soluble ions in PM2.5 and the differences under varying environmental conditions in western China are poorly understood. During the period from July 2021 to July 2022, we conducted synchronous offline sampling of PM2.5 by deploying four identical sampling devices at altitudes ranging from 1500 m to 3600 m along the unique topography of the Lanzhou River Valley in western China. The major chemical components in PM2.5 are carbonaceous aerosols, NO3− and SO42− with the average concentrations of 28.07, 8.66 and 6.01 μg m−3 for the urban and surrounding rural areas, and 9.94, 2.45 and 3.11 μg m−3 for background site of Lanzhou. The concentrations of OC (organic carbon) and EC (elemental carbon) gradually decrease from urban area to suburban sites with increasing elevation. Coal combustion, vehicle emissions, secondary formation, salted aerosols and surface dust resuspension are found to be the primary sources of PM2.5 in both urban and rural areas of Lanzhou. The contribution of vehicle emissions decreases from urban areas to mountainous rural areas, contrary to the coal combustion and biomass burning. The study is significant for taking targeted measures to solve air pollution problem at different environmental conditions (urban, rural and background) over complex terrain.