Fly ash from coal-fired power plant has been reported to be a promising sorbent for mercury capture. However, the nature of the Hg–fly ash interaction is still unknown. In this work, the thermal decomposition method is used to evaluate the effect of the fly ash composition on the retention of mercury in coal-combustion flue gas. Eight fly ash samples were collected from the electrostatic precipitators of eight full-scale pulverized coal-fired power plants. The high-temperature treatment is performed to gradually decompose the carbon and mercury species in the fly ashes, and each fly ash are divided into four groups. The content of the carbon and mercury present in all the examined samples was measured. The results show that the most mercury species are decomposed from the fly ash at 300°C. The quantity of carbon in the fly ash alone does not determine the amount of mercury retained at any temperature. The decomposed mercury content is related to the mercury content of the fly ash. The carbon particle with ultra-strong mercury capture capacity cannot be found in the fly ash. Furthermore, X-ray fluorescence spectrometry (XRF) is used to identify the elemental composition of inorganic components of all the native fly ashes. Multivariate linear regression method is used to assess the dominated components that determine the mercury retention capacity of these fly ashes. The results indicate that the carbon, silicon, iron, sulfur, and magnesium in fly ash all play a major role in mercury retention. Hence, both carbon and inorganic components influence the retention of mercury in the fly ash.