The effects of water on the structure and mechanical properties of a coal gangue-based geopolymer were investigated experimentally and compared with the results of molecular dynamics simulations. The chemical composition and crystalline phases of the raw materials were analyzed via X-ray fluorescence and X-ray diffraction (XRD) after 28 days of curing, and the mechanical properties were tested. Molecular structure models for H2O/Al ratios of 2.5, 3, and 3.5 were established using molecular dynamics simulation software, and structural optimization and molecular dynamics simulations were performed under the universal force field. Based on the obtained results, the energy and temperature change curves, simulated XRD spectrum, radial distribution function, hydrogen bonding, mean squared displacement, and mechanical properties of the amorphous geopolymer were analyzed. The rationality of the molecular model was verified through comparison with the results of the molecular dynamics simulations and experiments, and as the H2O/Al ratio increased within a given range, the energy and temperature stability time of the system are decreased, the area of the dispersion peak is decreased, the amorphous phase is increased, the diffusion degree of Si and Al in the system is decreased, the length of Si-O bond, Al-O bond, H-O bond and O-O bond is shortened, and the structure is more stable. The mechanical properties obtained from the experiment were also improved.