PurposeThis study aims to analyze the impact of fractional derivatives on heat transfer and entropy generation during transient free convection inside various complex porous enclosures, such as triangle, L-shape and square-containing wavy surfaces. These porous enclosures are saturated with Cu-water nanofluid and subjected to the influence of a uniform magnetic field.Design/methodology/approachIn the present study, Darcy’s model is used for the momentum transport equation in the porous matrix. Additionally, the Caputo time fractional derivative is introduced in the energy equation to assess the heat transfer phenomenon. Furthermore, the total entropy generation has been computed by combining the entropy generation due to fluid friction (Sff), heat transfer (Sht) and magnetic field (Smf). The complete mathematical model is further simulated using the penalty finite element method, and the Caputo time derivative term is approximated using the L1 scheme. The study is conducted for various ranges of the Rayleigh number (102≤Ra≤104), Hartmann number (0≤Ha≤20) and fractional order parameter (0<α<1) with respect to time.FindingsIt has been observed that the fractional order parameter α governs the characteristics of entropy generation and heat transfer within the selected range of parameters. The Bejan number associated with heat transfer (Beht), fluid friction (Beff) and magnetic field (Bemf) further demonstrate the dominance of flow irreversibilities. It becomes evident that the initial evolution state of streamlines, isotherms and local entropy varies according to the choice of α. Additionally, increasing Ra values from 102 to 104 shows that the heat transfer rate increases by 123.8% for a square wavy enclosure, 7.4% for a triangle enclosure and 69.6% for an L-shape enclosure. Moreover, an increase in the value of Ha leads to a reduction in heat transfer rates and entropy generation. In this case, Bemf→1 shows the dominance of the magnetic field irreversibility in the total entropy generation.Practical implicationsRecently, fractional-order models have been widely used to express numerous physical phenomena, such as anomalous diffusion and dispersion in complex viscoelastic porous media. These models offer a more accurate representation of physical reality that classical models fail to capture; this is why they find a broad range of applications in science and engineering.Originality/valueThe fractional derivative model is used to illustrate the flow pattern, heat transfer and entropy-generating characteristics under the influence of a magnetic field. Furthermore, to the best of the author’s knowledge, a fractional-derivative-based mathematical model for the entropy generation phenomenon in complex porous enclosures has not been previously developed or studied.