Magnetic skyrmions are topological spin textures and promising candidates for novel spintronic applications. Recent studies on the current-driven dynamics of ferromagnetic (FM) skyrmions revealed that they exhibit an undesirable transverse motion, the skyrmion Hall effect. For antiferromagnetic (AFM) skyrmions, a vanishing skyrmion Hall effect was predicted, along with faster dynamics. However, their zero net magnetization obstructs efficient detection. Ferrimagnetic (FI) materials promise to combine both advantages: fast, AFM-like dynamics and easy read-out via stray fields. Here, we investigate the current-driven and Brownian dynamics of skyrmions in a FI with a compensation point. We perform atomistic spin dynamics simulations based on a model Hamiltonian and the stochastic Landau-Lifshitz-Gilbert equation supplemented with spin-orbit torques, accompanied by analytical calculations based on a collective coordinate approach. Our results unveil a nonmonotonic temperature dependence of the velocities and the diffusion coefficient with a strong enhancement at the angular momentum compensation temperature, due to scaling from FM- to AFM-like dynamics. These findings open up a new pathway for the efficient manipulation of skyrmion dynamics via temperature.