Ensembles of magnetic skyrmions in confined geometries are shown to exhibit thermally driven motion on two different time scales. The intrinsic fluctuating dynamics (t ∼ 1 ps) are governed by short-range symmetric and antisymmetric exchange interactions, whereas the long-time limit (t ≳ 10 ns) is determined by the coaction of skyrmion–skyrmion-repulsion and the system’s geometry. Micromagnetic simulations for realistic island shapes and sizes are performed and analyzed, indicating the special importance of skyrmion dynamics at finite temperatures. We demonstrate how the competition between skyrmion mobility and observation time directly affects the addressability of skyrmionic bits, which is a key challenge on the path of developing skyrmion-based room-temperature applications. The presented quasiparticle Monte Carlo approach offers a computationally efficient description of the diffusive motion of skyrmion ensembles in confined geometries, like racetrack memory setups.