The hydration shells of phosphate ions and phosphate groups of nucleotides and phospholipid membranes display markedly different structures and hydrogen-bond strengths. Understanding phosphate hydration requires insight into the spatial arrangements of water molecules around phosphates and in thermally activated structure fluctuations on ultrafast time scales. Femtosecond two-dimensional infrared spectroscopy of phosphate vibrations, particularly asymmetric stretching vibrations between 1000 and 1200cm-1, and abinitio molecular dynamics (AIMD) simulations are combined to map and characterize dynamic local hydration structures and phosphate-water interactions. Phosphoric acid H3PO4 and its anions H2PO4-, HPO42-, and PO43- are studied in aqueous environments of different pH value. The hydration shells of phosphates providing OH donor groups in hydrogen bonds with the first water layer undergo ultrafast structural fluctuations, which induce a pronounced spectral diffusion of vibrational excitations on a sub-300fs time scale. With a decreasing number of phosphate OH groups, the hydration shell becomes more ordered and rigid. The 2D-IR line shapes observed with hydrated PO43- ions display a pronounced inhomogeneous broadening, reflecting a distribution of hydration geometries without fast equilibration. The AIMD simulations allow for an in-depth characterization of the hydration geometries with different numbers of water molecules in the first hydration layer and different correlation functions of the fluctuating electric field that the water environment exerts on the vibrational phosphate oscillators.