In this study, aluminum foil (AF) and expanded perlite (EP) were used as the reinforcing fillers to improve the ablation properties of phenyl methyl vinyl silicone rubber composites. The expansion and caramelization of char layer enhanced the ablation and thermal insulation performance of silicone rubber-based composites. The improved bonding of char layer provided a good basis for enhancing ablative properties. The presence of EP increased the emissivity of the char layer which improved the heat radiation properties during ablation process, and it was advantageous to lower surface temperature and reduce thermal erosion. The graphitic degree of char layer was increased which improved high-temperature and oxidative resistance. SEM observations revealed that the surface and cross-section of the char layer were densified, as characterized by the pore size distribution analysis. The above results indicated that the ablation and anti-erosion resistance as well as thermal insulation behavior of silicone rubbers were significantly improved by implementing the expansion and caramelization strategy, which exhibit promising applications as flexible anti-ablation materials for thermal protection purposes.