This study aims to investigate the effects of nanoalumina (nano-Al2O3, NA) and nanotitanium dioxide (nano-TiO2, NT) particles on the mechanical and microstructural properties of oil well cement composite pastes. Eleven groups of cement composite pastes mixed with NA (0.5–2.5 wt.%), NT (0.5–2.5 wt.%) particles and silica flour were designed by central composite response surface design method (CCD method). Meanwhile, a reference group (SF-OWC) consisting of oil well cement and silica flour was designed. These pastes were cured at 300 °C/13.0 MPa in an autoclave to simulate the downhole condition during the steam injection process in steam injection wells. The compressive strength and Young's modulus of cured cement composite pastes were tested, and mathematical models were developed to evaluating the response of NA and NT particle content on compressive strength and Young's modulus. Moreover, the mineralogy and microstructural evolution of the pastes were investigated using multi-technique methods. The results showed that the compressive strength of the mixed nanoparticles modified SF-OWC composite pastes increased by 7.5%–36.4% compared to the reference group. Moreover, this enhancement is more significant when the incorporated NA or NT particles dominate the mixed nanoparticles. The model prediction results show that adding 2.5 wt.% NA and 0.5 wt.% NT particles or adding 0.5 wt.% NA and 2.5 wt.% NT could increase the compressive strength of cement composite pastes by 58.8% and 37.5%, respectively. The mechanism analysis showed that when the incorporated mixed nanoparticles were mainly NT particles, the improvement of mechanical properties was mainly attributed to the filling effect and nuclear effect of NT particles. When the incorporated mixed nanoparticles were mainly NA particles, more C-(A)-S-H gels were retained and tightly filled and bonded with xonotlite, which made the microstructure of the cement composite pastes more dense, stronger bonding, and more conducive to resistance to load or stress variations.