In this article, turbulent heat transfer of nanofluid flow in square passage with protruded rib shape is numerically and experimentally studied over Reynolds number ranges of 4000–18000. Different nanoparticles (Al2O3, CuO, and ZnO), with different concentration (φ) range of 1–4% and different nanoparticle diameter (dnp) range of 30–45nm are disperse in water (base fluid). Several parameters such as stream wise distance (Xs/dp) range of 1.4–2.6, span wise distance (Ys/dp) range of 1.4–2.6, ratio of protruded height to print diameter (ep/dp) range of 0.83–1.67 also studied to find the consequence on thermal and hydrodynamic characteristics. Simulations were carried out to obtain heat and fluid flow behaviour of smooth and ribbed square channel using commercial CFD software, ANSYS 15.0 (Fluent). Renormalization k-ε model was employed to assess the influence of protruded ribs on turbulent flow and velocity field. The outcome indicates that Al2O3 nanofluid has the highest value of average Nusselt number as compare to other nanofluids. The average Nusselt number increases as the concentration increases and it decreases as nanoparticle diameter increases. The thermal hydrodynamic performance parameter based on equal pumping power, average Nusselt number and average friction factor were found to be highest for Al2O3, φ=0.04, dnp=30nm, Xs/dp=1.8, Ys/dp=1.8 and ep/dp=1.0. The numerical data are compared with the corresponding experimental data. Comparison between CFD and experimental analysis results showed that good agreement as the data fell within ±7.0% error band.