Approximants consisting of 1--165 Si atoms with complete fluorine (F), hydroxyl (OH), amino $({\text{NH}}_{2})$, methyl $({\text{CH}}_{3})$, and hydrogen (H) termination were investigated by density-functional-Hartree-Fock (DF-HF) ground-state (GS) calculations within the B3LYP hybride functional formalism using a 6--31G(d) Gaussian type molecular-orbital (MO) basis set. With increasing polarity of the Si interface bonds, the highest occupied MO (HOMO)-lowest unoccupied MO (LUMO)-gaps depend less on quantum confinement but more on the nature of the interface. The HOMO and LUMO positions are a function of the anion/functional group intended to model different dielectric matrices hosting the Si quantum dot (QD). We discuss the influence of the Si interface bond termination on charge transfer and local breakdown of the MO symmetry. Relating these observations to the ratio of interface anions to Si core atoms, we estimate a size range below which the electronic structure of Si QDs is determined exclusively by the nature of the interface bonds. For Si cores consisting of $\ensuremath{\ge}10$ Si atoms, the interface governs the electronic structure with quantum confinement competing for covalent and weak polar interface terminations or being only a secondary effect for strong polar interface terminations. We estimate that the interface has a major impact on the electronic structure of Si QDs in ${\text{Si}}_{3}{\text{N}}_{4}$, ${\text{SiO}}_{2}$, or fluorides to a Si core size of ca. 1330 atoms.