The Berry connection and curvature are key components of electronic structure calculations for atoms and molecules in magnetic fields. They ensure the correct translational behavior of the effective nuclear Hamiltonian and the correct center-of-mass motion during molecular dynamics in these environments. In this work, we demonstrate how these properties of the Berry connection and curvature arise from the translational symmetry of the electronic wave function and how they are fully captured by a finite basis set of London orbitals but not by standard Gaussian basis sets. This is illustrated by a series of Hartree-Fock calculations on small molecules in different basis sets. Based on the resulting physical interpretation of the Berry curvature as the shielding of the nuclei by the electrons, we introduce and test a series of approximations using the Mulliken fragmentation scheme of the electron density. These approximations will be particularly useful in ab initio molecular dynamics calculations in a magnetic field since they reduce the computational cost, while recovering the correct physics and up to 95% of the exact Berry curvature.