Cellulose is an important part of transformer insulation paper. Thermal aging of cellulose occurs in long-term operation of transformers, which deteriorates the mechanical properties and thermal stability of cellulose, resulting in a decrease in the transformer life. Therefore, improvement of the mechanical properties and thermal stability of cellulose has become a research hotspot. In this study, the effects of different silane coupling agents on the mechanical properties and thermal stability of modified cellulose were studied. The simulation results showed that the mechanical parameters of cellulose are only slightly improved by KH560 (γ-glycidyl ether oxypropyl trimethoxysilane) and KH570 (γ-methylacrylloxy propyl trimethoxy silane) modified nano-SiO2, while the mechanical parameters of cellulose are greatly improved by KH550 (γ-aminopropyl triethoxy silane) and KH792 (N-(2-aminoethyl)-3-amino propyl trimethoxy silane) modified nano-SiO2. The glass-transition temperature of the composite model is 24 K higher than that of the unmodified model. The mechanism of the change of the glass-transition temperature was analyzed from the point of view of free-volume theory. The main reason for the change of the glass-transition temperature is that the free volume abruptly changes, which increases the space for movement of the cellulose chain and accelerates the whole movement of the molecular chain. Therefore, modifying cellulose with KH792-modified nano-SiO2 can significantly enhance the thermal stability of cellulose.