Silica fume (SF) has practically become an essential phase in ultra-high performance concrete (UHPC) proportion due to its superior physical effect (filler effect) and chemical effect (pozzolanic reaction effect). However, the decoupling of these two effects was rarely investigated, which prevents a deeper comprehension of SF’s role in UHPC system. Accordingly, titanium dioxide (TD), an inert filler with similar particle size and shape to SF was selected as a comparison. Results showed that the preferential adsorption of SF on PCE molecules can shorten the induction period of UHPC pastes, and simultaneously benefited from the SF’s seeding effect, both of which can increase the cumulative heat release of UHPC pastes and promote the early cement hydration. In the later stage, SF has the possibility to reduce cement hydration, always exhibiting a lower promoting effect compared to TD. However, the pozzolanic reaction of SF dominates the sustained compressive strength development of UHPC by consuming CH to generate extra C-S-H with lower Ca/Si ratio, which reduces the non-homogeneity and connectivity of UHPC pores.