Isostructural ZrW2O8 and HfW2O8 show strong negative thermal expansion from 0.3 K up to their decomposition temperatures of approximately 1050 K. This behavior is especially unusual because these compounds are apparently cubic over their entire existence range. Detailed structural studies of ZrW2O8 were conducted using high-resolution neutron powder diffraction data taken at 14 temperatures from 0.3 to 693 K. Below 428 K, ZrW2O8 adopts the acentric space group P213 and has a well-ordered structure containing corner-sharing ZrO6 octahedra and two crystallographically distinct WO4 tetrahedra. Above the phase transition at 428 K, which appears to be second order, the space group becomes centric Pa3̄. The structure is now disordered with one oxygen site 50% occupied, suggesting the possibility of high oxygen mobility. Oxygen motion above 428 K is also suggested by dielectric and ac impedance measurements. The negative thermal expansion of ZrW2O8 and HfW2O8 is related to transverse thermal vibrations of bridging oxygen atoms. These lead to coupled rotations of the essentially rigid polyhedral building blocks of the structure. A semiquantitative model for both the negative thermal expansion and phase transition of these materials is proposed in light of the diffraction results.