The momentum distribution of particles accelerated at strong non-relativistic shocks may be influenced by the spatial distribution of the flow speed around the shock. This phenomenon becomes evident in the cosmic-ray-modified shock, where the particle spectrum itself determines the profile of the flow velocity upstream. However, the effects of a non-uniform flow speed downstream are unclear. Hydrodynamics indicates that the spatial variation of flow speed over the length scales involved in the acceleration of particles in supernova remnants (SNRs) could be noticeable. In the present paper, we address this question. We initially followed Bell's approach to particle acceleration and then also solved the kinetic equation. We obtained an analytical solution for the momentum distribution of particles accelerated at the cosmic-ray-modified shock with spatially variable flow speed downstream. We parameterised the downstream speed profile to illustrate its effect on two model cases: the test particle and non-linear acceleration. The resulting particle spectrum is generally softer in Sedov SNRs because the distribution of the flow speed reduces the overall shock compression accessible to particles with higher momenta. On the other hand, the flow structure in young SNRs could lead to harder spectra. The diffusive properties of particles play a crucial role as they determine the distance the particles can return from to the shock, and, as a consequence, the flow speed that they encounter downstream. We discuss a possibility that the plasma velocity gradient could be (at least partially) responsible for the evolution of the radio index and for the high-energy break visible in gamma rays from some SNRs. We expect the effects of the gradient of the flow velocity downstream to be prominent in regions of SNRs with higher diffusion coefficients and lower magnetic field, that is, where acceleration of particles is not very efficient.