In this study, Se substituted Co spinel ferrite nanoparticles (CoSe3xFe2–4xO4 (x ≤ 0.10), CoSe SFNPs) have been synthesized via sol-gel (SG) and hydrothermal (HT) routes and their structural, magnetic and morphological features have been compared. The effect of both the Se substitution and the method of synthesis on the structural, morphological and magnetic features of the host CoFe2O4 have been investigated. The structure and morphology of the SFNPs were examined using XRD, EDX, SEM, TEM and HR-TEM methods. XRD analysis confirmed the single-phase formation of CoSe SFNPs for both HT and SG routes. A TEM study for the HT process demonstrated the development of spherical nanoparticles with uniform size, whereas the SG procedure produced NPs with an indeterminate shape. By examining the hysteresis loops at 300 and 20 K and analysing the temperature-dependent changes in magnetization (M), the magnetic characteristics of CoSe (x ≤ 0.10) SFNPs prepared by the SG and HT methods were investigated. CoSe SFNPs possess magnetically soft and hard natures at 300 and 20 K, respectively. Under each synthesis condition, at 20 K, the SQR values are above 0.5, which signifies well-ordered magnetic domains at this temperature. Across all the materials, as the temperature decreases, zero field cooling (ZFC) branches exhibit a non-linear decline in magnetization which becomes nearly stable in the field cooling (FC) branches. The deviation of the ZFC curve with respect to FC one is more pronounced in the case of SFNPs prepared by the HT process. Our findings demonstrate that by employing the desired synthesis method, Se dopants can be applied to achieve particular magnetic characteristics of CoFe2O4 NPs.