Whereas the cybotactic theory and thermodynamic modelling have proved invaluable tools in understanding the structures and properties of alkali silicate glasses, questions have been raised as to their validity for alkaline earth and related glasses such as PbO–SiO2. The present paper discusses the specific cases of the CaO–SiO2 and PbO–SiO2 systems, and it is shown that the presence of Si[3] units (silicate tetrahedra with three bridging and one nonbridging oxygen atoms) can easily be explained in terms of the thermodynamic equilibria that underlie the model of associated solutions and the cybotactic theory. Similarly, the much more random distribution of silicate tetrahedral species in PbO–SiO2 glasses derives from the amphoteric nature of PbO. A related question concerns the relevance of atomistic structural modelling/simulation to the evaluation of structural theories of glasses, but to date all of the models of binary silicate glasses have been generated using periodic boundary conditions, which means that they are incapable of reproducing the long range disorder that characterises the vitreous state. Furthermore, it is demonstrated that, in their present form, the RMC and related computer codes, such as EPSR, are fundamentally flawed, in that they merely involve the fitting of an early crystallite model to experimental diffraction data, albeit one where the average internal structure of the crystallites is based on a large highly disordered unit cell, but the crystallites themselves have an entirely unphysical shape. It is also concluded that, to fully interpret the structure of binary and multicomponent glasses, it is essential to study the relevant phase diagram, together with the structures of the thermodynamically-stable and metastable crystalline phases that occur in that particular glass-forming system, and to understand that, since the supercooled liquid is only transiently metastable, the cybotactic/chemical grouping species present in the final glass may not necessarily be determined by equilibrium thermodynamics, but may be greatly influenced by the quench rate. The temperature range over which these crystalline phases/polymorphs are stable is also important, as is the temperature dependence of the glass transition temperature, Tg, and its relationship to the solidus temperature, Ts, at the same composition. Only in this way is it possible to derive the maximum information concerning the structure of a given glass and, much more importantly, to explain why this glass has its particular structure. It is therefore concluded that the key to developing a comprehensive theory of the formation and structure of the vitreous state lies not with ever more precise determinations of the short range order (i.e. diffraction studies), but rather in understanding the role of the thermodynamic equilibria that drive the characteristic long wavelength fluctuations in both number density and composition that distinguish the vitreous from the crystalline state.