The $g$ tensor, which describes the interaction energy of the spin with an external magnetic field, plays an essential role in controlling and manipulating single spins in semiconductor quantum dots. Spin-correlated orbital currents can strongly affect the $g$ tensor. This paper investigates theoretically and experimentally how these currents depend on the shape of quantum dots and how they affect the anisotropy of the electron $g$ tensor. It is found that the spin-correlated orbital currents form a simple current loop perpendicular to the magnetic moment's orientation, and are therefore directly sensitive to the shape of the nanostructure. This simple and intuitive picture is validated by a systematic experimental magnetoluminescence study of the size dependence of the separate electron and hole $g$ tensors of InAs/InP quantum dots.