Polymer templates are important additives for electrospun ceramic nanofibers and have strong relationship to its properties. In this study, polyvinylpyrrolidone (PVP) and polyvinyl butyral (PVB) were used as templates to prepare mullite nanofibers. The morphology, microstructure and mechanical properties of the nanofibers were investigated and compared. Results show that after 1000 °C calcination, fibers prepared using PVB had a porous structure with surface area of 40.11 m2/g, and were composed of mullite crystals with elastic modulus of ∼27.5 GPa. However, the fibers prepared using PVP had a dense structure with surface area of 8.14 m2/g, and were composed of γ-Al2O3 and mullite crystals with a higher elastic modulus of ∼79.4 GPa. The influence mechanism was confirmed that the different binding states of polymer and aluminasilica precursor were the main reason for the fiber differences. These results give a reference for polymer template categories selection for preparing mullite nanofibers applied in different areas like filtration, catalysis and insulation.