A hydrothermal method followed by heat treatment was used to synthesize Mn-substituted Ba2Co2−x Mn x Fe12O22 nanoparticles with a nominal chemical composition of $$ 0 \le x < 1 $$ and step gap of 0.3. In this study, the effect of Mn substitution on Co2Y-type barium hexaferrite is investigated after employing x-ray diffraction for crystal structure, field emission scanning electron microscopy for morphology, energy dispersive analysis of x-ray spectroscopy for elemental composition, Fourier transform infrared spectroscopy to confirm bond modes, and vibrating sample magnetometry for magnetic measurements. It was found that the sample at x = 0.9 is of particular interest due to its large coercivity and anisotropy. Later on, for x = 0.9, temperature-dependent magnetic analyses including hysteresis loops, zero-field-cooled, and field-cooled at a particular field of 100 Oe were performed. The decreasing trend in saturation magnetization with increase in temperature was estimated. On the other hand, first an increase and then decrease in coercivity values were observed. These loops also revealed dependence of coercivity on magneto-crystalline anisotropy and average crystallite size of nanoparticles. Dielectric measurements at x = 0.9 make it suitable for high frequency applications.