This study focused on the effect of partial substitution of Ta2O5 by Nb2O5 on the glass formation, structure and electrocial properties of Li2O-Ta2O5-ZrO2-SiO2 glasses. Structural analysis showed that Nb5+ enters the glass matrix in the form of NbO6 octahedra. When the content of Nb2O5 is 2.94 mol%, Nb5+ ions disrupt the silicate lattice and act as network modifiers. When the content of Nb2O5 is 5.88 and 11.77 mol% , Nb5+ ions play the role of glass formers, forming a three-dimensional chain lattice by interconnecting the NbO6 octahedra with different deformations. Among all the glass samples, the glass containing 5.88 mol% Nb2O5 has the maximum ionic conductivity of 1.066 × 10−6 S /cm at 250 °C, which is an order of magnitude greater than the ionic conductivity of the matrix glass. These glass samples have the potential to be used as solid electrolytes in electrochemical applications.