This study presents a novel investigation into the thermal behavior of a hybrid nanofluid (MgO − Ag − H2O) in natural convection around a porous fin in a triangular enclosure under the influence of radiation and magnetohydrodynamic effects. The research uniquely combines these complex phenomena, addressing a significant gap in the literature. This configuration has potential applications in advanced solar thermal collectors, electronic cooling systems for high-power devices, and compact heat exchangers in various industries. The main objectives are to understand how various parameters influence heat transfer and fluid flow behavior and to optimize the design for enhanced thermal performance. The study considers a range of variables including Rayleigh number (103 − − 106), Hartmann number (0 – 50), Aspect ratio (0.3 – 0.6), radiation parameters (Rd = 1 − − 5, λ = 1 − 5), and volume concentration (0- 0.05), which have been numerically analyzed using the finite element method (FEM). The findings reveal that increasing the Darcy number (Da) enhances heat transfer at low Rayleigh numbers (Ra = 103,104). However, at higher Ra (Ra = 106), the impact of Da becomes more complex, with a critical Da beyond which heat transfer efficiency decreases due to an increase in flow resistance. The nanoparticle volume concentration plays a vital role, as higher concentrations lead to improved heat transfer efficiency, especially at higher Ra, through enhanced thermal conductivity and thermal dispersion. The length of the porous fin greatly impacts fluid flow patterns and heat transfer rates, with longer fins creating more complex flow patterns, promoting enhanced heat transfer and stronger thermal plumes. Thermal radiation, represented by the radiation parameters (Rd and λ), significantly influences both the heat transfer rate and the convective flow patterns within the enclosure. This study also incorporates a comprehensive entropy generation analysis, providing novel insights into system irreversibilities and optimization potential. The entropy analysis reveals the complex interplay between various parameters and their impact on system efficiency, offering valuable guidance for designing high-performance thermal management systems.