This study explores the application of Al2O3-ZnO-Fe3O4/Water ternary nanofluids for enhanced thermal management in engineering applications, addressing the MHD Darcy–Forchheimer problem with an emphasis on optimizing cooling processes in a 2D-wavy cavity under various moving boundary conditions. Aimed at surpassing the limitations of conventional cooling fluids, the research employs the finite volume method within the OpenFOAM® framework to investigate the impact of nanofluid integration on heat transfer efficiency and entropy generation, utilizing partial heating and eight distinct boundary scenarios. By comparing the performance of ternary nanofluids against that of hybrid and mono nanofluids, we identify optimal conditions that significantly enhance cooling effectiveness, as evidenced by a 37% improvement in heat exchange efficiency over water. The findings demonstrate the superior performance of Al2O3-ZnO-Fe3O4/Water nanofluids in achieving higher average Nusselt numbers at low Darcy numbers, despite an increase in entropy. Optimal conditions were identified for the most favorable scenario involving the movement of cavity lids, based on the Nusselt number, specifically in Case 7 (ul1+ and ul2−). Decreasing the number of waves in the cavity enhances heat transfer and reduces irreversibility. Finally, the study shows that the influence of lid motion on skin friction significantly varies with Richardson numbers, and the use of nanofluids over pure water notably reduces skin friction.