We study the diffusive motion of a particle in a subharmonic potential of the form U(x) = |x| c (0 < c < 2) driven by long-range correlated, stationary fractional Gaussian noise ξ α (t) with 0 < α ⩽ 2. In the absence of the potential the particle exhibits free fractional Brownian motion with anomalous diffusion exponent α. While for an harmonic external potential the dynamics converges to a Gaussian stationary state, from extensive numerical analysis we here demonstrate that stationary states for shallower than harmonic potentials exist only as long as the relation c > 2(1 − 1/α) holds. We analyse the motion in terms of the mean squared displacement and (when it exists) the stationary probability density function. Moreover we discuss analogies of non-stationarity of Lévy flights in shallow external potentials.