ABSTRACT The detection of Intermediate-Mass Black Holes (IMBHs) in dwarf galaxies is crucial to closing the gap in the wide mass distribution of black holes ($\sim 3 \, {\rm M_\odot }$ to $\sim 5 \times 10^{10} \, {\rm M_\odot }$). IMBHs originally located at the centre of dwarfs that later collide with the Milky Way (MW) could be wandering, undetected, in our Galaxy. We used TNG50, the highest resolution run of the IllustrisTNG project, to study the kinematics and dynamics of star clusters, in the appropriate mass range, acting as IMBH proxies in an MW analogue galaxy. We showed that $\sim 87{{\ \rm per\ cent}}$ of our studied IMBHs drift inward. The radial velocity of these sinking IMBHs has a median magnitude of $\sim 0.44 \, \mathrm{ckpc \, h^{-1} \, Gyr^{-1}}$ and no dependence on the black hole mass. The central $1 \, \rm ckpc \, h^{-1}$ has the highest number density of IMBHs in the galaxy. A physical toy model with linear drag forces was developed to explain the orbital circularization with time. These findings constrain the spatial distribution of IMBHs, suggesting that future searches should focus on the central regions of the Galaxy. Additionally, we found that the 3D velocity distribution of IMBHs with respect to the galactic centre has a mean of $\sim 180 \, \mathrm{km \, s^{-1}}$ and larger variance with decreasing radius. Remarkably, the velocity distribution relative to the local gas shows significantly lower values, with a mean of $\sim 88 \, \mathrm{km \, s^{-1}}$. These results are instrumental for predicting the accretion and radiation properties of IMBHs, facilitating their detection with future surveys.