The submicroscopic structure of lithium niobium silicate glasses of the compositions 2xLiNbO3 · (1 − x)(Li2O · 2SiO2) (x = 0.40, 0.45, 0.50) and 30Li2O · 25Nb2O5 · 45SiO2 in the initial state and after heat treatment for different times at temperatures in the vicinity of the glass transition point T g are investigated using X-ray powder diffraction, small-angle neutron scattering (SANS), synchrotron small-angle X-ray scattering (SAXS), and electron microscopy. A nanostructure with inhomogeneities ∼40 A in size is formed in glasses at the initial stages of phase separation at temperatures in the range 600–670°C. This structure is responsible for the appearance of the second-order optical nonlinearity. The SANS, SAXS, and electron microscopic data on the inhomogeneity size are in good agreement with each other. According to the X-ray diffraction, SANS, and SAXS data, the ordering of the glass structure and the difference between the density of inhomogeneities and the density of the matrix increase in the course of heat treatment. At the initial stage of amorphous phase separation, the glass decomposes into regions enriched in SiO2 and regions with an increased content of lithium and niobium. An increase in the temperature or time of heat treatment results in the precipitation of LiNbO3 ferroelectric crystals. The results obtained allow us, for the first time, to make the inference that nanoscale changes in the glass structure lead to considerable changes (by one order of magnitude and more) in the quadratic optical nonlinearity, which can be controlled by heat treatment. The origin of the second-order optical nonlinearity is associated with both the nanosized modulations of the polarizability due to the inhomogeneous glass structure and the polarity of structural nanoinhomogeneities from which the LiNbO3 phase precipitates at the later stages of phase separation.