Biomass-derived hard carbons are promising anode materials of sodium-ion batteries (SIBs) due to their low cost, renewability and high specific capacity. However, there are inevitably some impurities in biomasses that may not only affect the purity, but microstructure and electrochemical performance of hard carbons. Although the impurities can be removed by appropriate treatments after carbonization, the microstructure of hard carbons can hardly be changed. In this work, taking ficus macrophylla leaves (FMLs) as the model biomass, the effects of impurities and their removing process on the microstructure and Na ion storage performance of hard carbons are investigated. It is found that the presence of Ca and Si-bearing impurities in FMLs results in more graphite-like structure in hard carbons because of their catalytic graphitization effect. The acid and base washing processes after carbonization can mostly get rid of the inorganic impurities is unable to increase the plateau region capacities associating to Na ion insertion in the interlayer of graphitic microcrystals and Na metal filling in micropores. By contrast, by stepwisely removing the Ca and Si-bearing impurities from raw materials before carbonization, hard carbons with high purity, larger interlayer space and more micropores are produced, which exhibit a high reversible specific capacity of 336 mAh g−1 at 20 mA g−1, a high initial Coulombic efficiency of 93 %, excellent rate capability (245 mAh g−1 at 2 A g−1) and robust cycle stability. This study implies that controlling the impurity removal process can simultaneously tune the microstructure of hard carbon anodes derived from biomasses towards high performance Na ion storage.