Vanadium-doped barium hexaferrite with a nanostructure is a highly valuable material in various technological fields, such as electronics, permanent magnets, and sensors. The Ba1-xFe12-xVxO19 (x = 0, 0.02, 0.04, 0.06, 0.08, and 0.1) nanoparticles derived from the aqueous solutions containing Fe:Ba molar ratio of 10:1 through the sol-gel auto-combustion method. Computational study was also performed using the first-principles density functional theory (DFT) approach. Crystal structure optimization, band structure, and density of states (DOS) calculations were conducted by CASTEP code. The variation of the structural, magnetic, optical, morphological, and electronic properties of V5+-doped barium hexaferrite was investigated. The XRD analysis combined with the Rietveld refinement showed the hexagonal structure of M-type barium ferrite (BaM), confirmed by the FT-IR analysis. The morphology of BaM nanoparticles was studied by the FE-SEM and TEM micrographs. In addition, magnetic and optical properties were analyzed through VSM and UV–Vis analysis. Crystallite size was found to be highly effective in tuning the coercivity and optical band gap of barium hexaferrites, which varied, respectively, from 3.24 to 4.83 kOe, and 2.69–3.69 eV. Magnetic results showed that several variables like cations distribution, lattice strain, and hematite secondary phase affected the nanoparticles’ magnetization. The DFT simulation results showed a sharp reduction of electronic band gap energy whether V takes the position of Ba or Fe (from 1.10 eV in undoped to 0.64 and 0.14 eV in doped structures). The projected density of states (PDOS) calculations demonstrated that the d orbitals of V and Ba mainly contribute to the valence band maximum (VBM) and conduction band minimum (CBM), respectively.