We implement and benchmark the frozen core approximation, a technique commonly adopted in electronic structure theory to reduce the computational cost by means of mathematically fixing the chemically inactive core electron states. The accuracy and efficiency of this approach are well controlled by a single parameter, the number of frozen orbitals. Explicit corrections for the frozen core orbitals and the unfrozen valence orbitals are introduced, safeguarding against seemingly minor numerical deviations from the assumed orthonormality conditions of the basis functions. A speedup of over twofold can be achieved for the diagonalization step in all-electron density-functional theory simulations containing heavy elements, without any accuracy degradation in terms of the electron density, total energy, and atomic forces. This is demonstrated in a benchmark study covering 103 materials across the Periodic Table and a large-scale simulation of CsPbBr3 with 2560 atoms. Our study provides a rigorous benchmark of the precision of the frozen core approximation (sub-meV per atom for frozen core orbitals below -200 eV) for a wide range of test cases and for chemical elements ranging from Li to Po. The algorithms discussed here are implemented in the open-source Electronic Structure Infrastructure software package.