In this study, we synthesize pristine and (Mg, Fe) co-doped SnO2 nanoparticles using the chemical co-precipitation method, where the Mg doping amount is fixed (2 mol%) and the amount of Fe is varied between 2 and 6 mol%. The narrow and sharp natures of intense XRD peaks notify that the crystallinity is pretty well. The formation of the single-phase tetragonal rutile type structure of all prepared compounds has been identified by Rietveld refinement, Fourier-transform infrared spectroscopy (FTIR), and Raman spectroscopy. Moreover, the well incorporation of Fe and Mg in SnO2 and absence of any other unexpected elements are confirmed by the energy dispersive X-ray spectroscopy. How the co-doping of Mg and Fe alter the crystallite size, microstrain, and dislocation density of the prepared samples have been investigated by Scherrer method with various modified version of it, Williamson-Hall method (W–H) in three factions (UDM, USDM, and UDEDM), Size strain plot (SSP) method, Halder-Wagner (H–W) method, Wagner-Aqua (W-A) method, and Warren-Averbach (WA) method. The obtained crystallite size using the WA method is smaller than the other methods and Scherrer-based methods show much comparable (except SLMSE) sizes. Additionally, the W–H, H–W, SSP, and W-A plots show almost similar tendency of decreasing crystal size, increasing dislocation density and lattice strain with increasing doping. The scanning electron microscopy (SEM) profiles revealed that the synthesized nanoparticles are agglomerated in nature, where the agglomeration increases with decreasing crystal size. The magnetic response of the prepared samples revealed a weak ferromagnetic (or soft magnetic) nature, where saturation magnetization increased due to doping, and a reducing trend of remanent magnetization and coercivity was explored which indicates the phase transition from ferromagnetic to paramagnetic. However, this soft magnetic nature was modified with the influence of defects like changes in lattice parameters, strain, and oxygen vacancy. This phase transition at higher doping concentration makes our nanomaterial as a suitable contender for memory devices, hyperthermia treatment, and photothermal effect.