This work focuses on the examination of the magneto-thermosolutal convection in a lid-driven wavy trapezoidal enclosure in the presence of thermal radiation. The wavy cavity is filled with radiative Fe3O4-Cu-H2O hybrid nanoliquid. In this work, two cases are considered depending on the location of heat and concentration sources. A portion of the vertical walls are kept hot and in high concentration while the remaining parts are in adiabatic condition. The adiabatic flat upper lid is moving towards the right with equal speed. In addition, the lower wavy border is cold and low concentration. The governing Navier–Stokes equations are modeled to describe thermosolutal phenomena within the wavy enclosure. These equations are solved by reconstructing a recently developed compact scheme. Computed outcomes are presented in terms of streamlines, isotherms, iso-concentrations, average Nusselt and Sherwood numbers to evoke the thermosolutal phenomena for various physical parameters. Also, computing in-house code is validated with the published numerical and experimental and literatures. This investigation surveys the roles of several well-defined parameters such as Richardson number (Ri), Lewis number (Le), Buoyancy ratio number (N), Radiation parameter (Rd), Hartmann number (Ha), inclination of angle (γ), undulation of the wavy surface (d) and solid volume fraction (ϕhnp) of the hybrid nanofluid. An increase in the Lewis number (Le) enhances species diffusion within the system but diminishes thermal transport. This work reveal that a change in ϕhnp from 0% to 4%, heat transfer is upgraded up to 4.28% in Case I and 3.93% in Case II while mass transfer is declined by 2.24% in Case I and 1.60% in Case II. Results indicate that Case I performed better than Case II in the case of energy and solutal transfer. This work has many practical applications such as heat exchangers, electronic device cooling, food processing and porous industrial processes.