In this study, finite element (FE) analyses based on the arbitrary Lagrangian–Eulerian technique were used to investigate the complex deformation behavior and numerous process variables that arise during the self-piercing riveting (SPR) joining of multi-materials, including carbon-fiber-reinforced polymer, GA590DP, and Al5052-H32 sheets in two- and three-layer combinations. Tensile tests and inverse analysis were conducted to determine the mechanical responses of each joining material, including the rivet, and equivalent strain-based fracture modeling approaches were adopted for a simple but effective crack initiation prediction during the mechanical joining process. The damage distribution and joining characteristics during the SPR joining were then numerically investigated by considering different die profiles. The high predictability of the proposed FE modeling approach in terms of the cross-sectional profile with joint quality indicators, such as the interlock, residual distance between the rivet leg and die, minimum thickness of the bottom sheet, load history, and peak load during mechanical joining, was validated through comparisons between the experiments and FE simulations.