Understanding the properties of transition metal materials requires consideration of the competition between the various types of electrons that comprise the system, whether they be delocalized and metallic, localized and magnetic, or localized and bonded. Here we report the refined crystal structure determination and physical properties of the two-dimensional layered compound TaOCl2. Within each layer we find a strong in-plane dimerization of Ta atoms perpendicular to the Ta-O-Ta chains, with significant disorder in the positioning of dimers in adjacent layers, suggesting that the free electron of each Ta localizes in Ta-Ta bonds. This is supported by resistivity measurements (ρ300K = 4.4*108 Ω-cm) as well as featureless, temperature-independent magnetic susceptibility up to T = 750 K. Analysis of the phonon contribution to specific heat shows the presence of a localized Einstein mode with TE = 175 K, attributed to the localized vibrations of the Ta-Ta dimers. Density functional theory calculations show that the magnitude of the Ta displacements (0.24 Å) has a dramatic effect on the electronic structure, changing from a metal with a bandwidth of 3.2 eV to an insulator with a flat band just below EF. Our results show how direct exchange interactions between Ta atoms that are relatively far from each other can still play a central role in the behavior of 5d transition metal compounds, taking advantage of the extended 5d atomic orbitals, and produce localized dimers with a bond distance of approximately 2.84 Å.