The recent studies focus on optimizing conditions to maximize the heat transfer rate while minimizing energy consumption, a key goal for various industries in both production processes and Automotive industry efficiency. Hybrid nanofluids play a crucial role in improving heat transfer phenomena. This study examines the enhanced heat transfer properties of a water-based hybrid nanofluid containing Al2O3 and SiO2 nanoparticles as it flows through a nonlinear stretching sheet in a porous medium. The convective flow is further influenced by a magnetic field and thermal radiation, adding complexity to the flow dynamics. To simplify the analysis, dimensional physical quantities are converted to non-dimensional parameters using appropriate similarity rules. These transformed equations are then numerically solved using the bvp4c function in MATLAB. To achieve optimal heat transfer rates, a robust statistical method known as response surface methodology (RSM) is employed, and validation is performed through an analysis of variance. Graphical representation is used to examine parametric behaviour, and a brief physical description is provided. Key findings include: the hybrid nanofluid shows a 10.61% increase in heat transfer rate compared to the base fluid at R = 0.2; the drag force coefficient for the Al2O3/H2O nanofluid is reduced by 13.98% compared to other hybrid nanofluids; and an increase in the nonlinear stretching sheet parameter improves velocity distribution while reducing temperature distribution. This study can be used in automotive transmission systems to enhance heat dissipation and lubrication, improving the efficiency and durability of the transmission.