The Badain Jaran Desert (BJD) in China is a desert with impressive sand dunes and a groundwater situation that has attracted numerous researchers. This paper gives an overview of the mysteries of groundwater in the BJD that are exhibited as five key problems identified in previous studies. These problems relate to the origin of the groundwater, the hydrological connection between the BJD and the Heihe River Basin (HRB), the infiltration recharge, the lake–groundwater interactions, and the features of stable isotope analyses. The existing controversial analyses and hypotheses have caused debate and have hindered effective water resources management in the region. In recent years, these problems have been partly addressed by additional surveys. It has been revealed that the Quaternary sandy sediments and Neogene-Cretaceous sandstones form a thick aquifer system in the BJD. Groundwater flow at the regional scale is dominated by a significant difference in water levels between the surrounding mountains and lowlands at the western and northern edges. Discharge of groundwater from the BJD to the downstream HRB occurs according to the regional flow. Seasonal fluctuations of the water level in lakes are less than 0.5 m due to the quasi-steady groundwater discharge. The magnitude of infiltration recharge is still highly uncertain because significant limitations existed in previous studies. The evaporation effect may be the key to interpreting the anomalous negative deuterium-excess in the BJD groundwater. Further investigations are expected to reveal the hydrogeological conditions in more detail.